Date: November 2016
Contact: Emil Velikov <emil.l.velikov@gmail.com>
Description:
- This file contains the revision field of the the PCI device.
+ This file contains the revision field of the PCI device.
The value comes from device config space. The file is read only.
The Linux kernel security team can be contacted by email at
<security@kernel.org>. This is a private list of security officers
who will help verify the bug report and develop and release a fix.
-It is possible that the security team will bring in extra help from
-area maintainers to understand and fix the security vulnerability.
+If you already have a fix, please include it with your report, as
+that can speed up the process considerably. It is possible that the
+security team will bring in extra help from area maintainers to
+understand and fix the security vulnerability.
As it is with any bug, the more information provided the easier it
will be to diagnose and fix. Please review the procedure outlined in
-admin-guide/reporting-bugs.rst if you are unclear about what information is helpful.
-Any exploit code is very helpful and will not be released without
-consent from the reporter unless it has already been made public.
+admin-guide/reporting-bugs.rst if you are unclear about what
+information is helpful. Any exploit code is very helpful and will not
+be released without consent from the reporter unless it has already been
+made public.
Disclosure
----------
to a few weeks. As a basic default policy, we expect report date to
disclosure date to be on the order of 7 days.
+Coordination
+------------
+
+Fixes for sensitive bugs, such as those that might lead to privilege
+escalations, may need to be coordinated with the private
+<linux-distros@vs.openwall.org> mailing list so that distribution vendors
+are well prepared to issue a fixed kernel upon public disclosure of the
+upstream fix. Distros will need some time to test the proposed patch and
+will generally request at least a few days of embargo, and vendor update
+publication prefers to happen Tuesday through Thursday. When appropriate,
+the security team can assist with this coordination, or the reporter can
+include linux-distros from the start. In this case, remember to prefix
+the email Subject line with "[vs]" as described in the linux-distros wiki:
+<http://oss-security.openwall.org/wiki/mailing-lists/distros#how-to-use-the-lists>
+
+CVE assignment
+--------------
+
+The security team does not normally assign CVEs, nor do we require them
+for reports or fixes, as this can needlessly complicate the process and
+may delay the bug handling. If a reporter wishes to have a CVE identifier
+assigned ahead of public disclosure, they will need to contact the private
+linux-distros list, described above. When such a CVE identifier is known
+before a patch is provided, it is desirable to mention it in the commit
+message, though.
+
Non-disclosure agreements
-------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure']
# The name of the math extension changed on Sphinx 1.4
if major == 1 and minor > 3:
Only for drivers with target_index() and CPUFREQ_ASYNC_NOTIFICATION unset.
get_intermediate should return a stable intermediate frequency platform wants to
-switch to, and target_intermediate() should set CPU to to that frequency, before
+switch to, and target_intermediate() should set CPU to that frequency, before
jumping to the frequency corresponding to 'index'. Core will take care of
sending notifications and driver doesn't have to handle them in
target_intermediate() or target_index().
- hpd-gpios: The GPIO pin for HDMI hotplug detect (if it doesn't appear
as an interrupt/status bit in the HDMI controller
itself). See bindings/pinctrl/brcm,bcm2835-gpio.txt
+- dmas: Should contain one entry pointing to the DMA channel used to
+ transfer audio data
+- dma-names: Should contain "audio-rx"
Required properties for DPI:
- compatible: Should be "brcm,bcm2835-dpi"
--- /dev/null
+Parallel to LVDS Encoder
+------------------------
+
+This binding supports the parallel to LVDS encoders that don't require any
+configuration.
+
+LVDS is a physical layer specification defined in ANSI/TIA/EIA-644-A. Multiple
+incompatible data link layers have been used over time to transmit image data
+to LVDS panels. This binding targets devices compatible with the following
+specifications only.
+
+[JEIDA] "Digital Interface Standards for Monitor", JEIDA-59-1999, February
+1999 (Version 1.0), Japan Electronic Industry Development Association (JEIDA)
+[LDI] "Open LVDS Display Interface", May 1999 (Version 0.95), National
+Semiconductor
+[VESA] "VESA Notebook Panel Standard", October 2007 (Version 1.0), Video
+Electronics Standards Association (VESA)
+
+Those devices have been marketed under the FPD-Link and FlatLink brand names
+among others.
+
+
+Required properties:
+
+- compatible: Must be "lvds-encoder"
+
+Required nodes:
+
+This device has two video ports. Their connections are modeled using the OF
+graph bindings specified in Documentation/devicetree/bindings/graph.txt.
+
+- Video port 0 for parallel input
+- Video port 1 for LVDS output
+
+
+Example
+-------
+
+lvds-encoder {
+ compatible = "lvds-encoder";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ lvds_enc_in: endpoint {
+ remote-endpoint = <&display_out_rgb>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ lvds_enc_out: endpoint {
+ remote-endpoint = <&lvds_panel_in>;
+ };
+ };
+ };
+};
--- /dev/null
+Drivers for the second video output of the GE B850v3:
+ STDP4028-ge-b850v3-fw bridges (LVDS-DP)
+ STDP2690-ge-b850v3-fw bridges (DP-DP++)
+
+The video processing pipeline on the second output on the GE B850v3:
+
+ Host -> LVDS|--(STDP4028)--|DP -> DP|--(STDP2690)--|DP++ -> Video output
+
+Each bridge has a dedicated flash containing firmware for supporting the custom
+design. The result is that, in this design, neither the STDP4028 nor the
+STDP2690 behave as the stock bridges would. The compatible strings include the
+suffix "-ge-b850v3-fw" to make it clear that the driver is for the bridges with
+the firmware specific for the GE B850v3.
+
+The hardware do not provide control over the video processing pipeline, as the
+two bridges behaves as a single one. The only interfaces exposed by the
+hardware are EDID, HPD, and interrupts.
+
+stdp4028-ge-b850v3-fw required properties:
+ - compatible : "megachips,stdp4028-ge-b850v3-fw"
+ - reg : I2C bus address
+ - interrupt-parent : phandle of the interrupt controller that services
+ interrupts to the device
+ - interrupts : one interrupt should be described here, as in
+ <0 IRQ_TYPE_LEVEL_HIGH>
+ - ports : One input port(reg = <0>) and one output port(reg = <1>)
+
+stdp2690-ge-b850v3-fw required properties:
+ compatible : "megachips,stdp2690-ge-b850v3-fw"
+ - reg : I2C bus address
+ - ports : One input port(reg = <0>) and one output port(reg = <1>)
+
+Example:
+
+&mux2_i2c2 {
+ status = "okay";
+ clock-frequency = <100000>;
+
+ stdp4028@73 {
+ compatible = "megachips,stdp4028-ge-b850v3-fw";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ reg = <0x73>;
+
+ interrupt-parent = <&gpio2>;
+ interrupts = <0 IRQ_TYPE_LEVEL_HIGH>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ stdp4028_in: endpoint {
+ remote-endpoint = <&lvds0_out>;
+ };
+ };
+ port@1 {
+ reg = <1>;
+ stdp4028_out: endpoint {
+ remote-endpoint = <&stdp2690_in>;
+ };
+ };
+ };
+ };
+
+ stdp2690@72 {
+ compatible = "megachips,stdp2690-ge-b850v3-fw";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ reg = <0x72>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ stdp2690_in: endpoint {
+ remote-endpoint = <&stdp4028_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ stdp2690_out: endpoint {
+ /* Connector for external display */
+ };
+ };
+ };
+ };
+};
====================
Required properties:
-- compatible: Should be "fsl,<chip>-ipu"
+- compatible: Should be "fsl,<chip>-ipu" where <chip> is one of
+ - imx51
+ - imx53
+ - imx6q
+ - imx6qp
- reg: should be register base and length as documented in the
datasheet
- interrupts: Should contain sync interrupt and error interrupt,
in this order.
- resets: phandle pointing to the system reset controller and
reset line index, see reset/fsl,imx-src.txt for details
+Additional required properties for fsl,imx6qp-ipu:
+- fsl,prg: phandle to prg node associated with this IPU instance
Optional properties:
- port@[0-3]: Port nodes with endpoint definitions as defined in
Documentation/devicetree/bindings/media/video-interfaces.txt.
};
};
+Freescale i.MX PRE (Prefetch Resolve Engine)
+============================================
+
+Required properties:
+- compatible: should be "fsl,imx6qp-pre"
+- reg: should be register base and length as documented in the
+ datasheet
+- clocks : phandle to the PRE axi clock input, as described
+ in Documentation/devicetree/bindings/clock/clock-bindings.txt and
+ Documentation/devicetree/bindings/clock/imx6q-clock.txt.
+- clock-names: should be "axi"
+- interrupts: should contain the PRE interrupt
+- fsl,iram: phandle pointing to the mmio-sram device node, that should be
+ used for the PRE SRAM double buffer.
+
+example:
+
+pre@21c8000 {
+ compatible = "fsl,imx6qp-pre";
+ reg = <0x021c8000 0x1000>;
+ interrupts = <GIC_SPI 90 IRQ_TYPE_EDGE_RISING>;
+ clocks = <&clks IMX6QDL_CLK_PRE0>;
+ clock-names = "axi";
+ fsl,iram = <&ocram2>;
+};
+
+Freescale i.MX PRG (Prefetch Resolve Gasket)
+============================================
+
+Required properties:
+- compatible: should be "fsl,imx6qp-prg"
+- reg: should be register base and length as documented in the
+ datasheet
+- clocks : phandles to the PRG ipg and axi clock inputs, as described
+ in Documentation/devicetree/bindings/clock/clock-bindings.txt and
+ Documentation/devicetree/bindings/clock/imx6q-clock.txt.
+- clock-names: should be "ipg" and "axi"
+- fsl,pres: phandles to the PRE units attached to this PRG, with the fixed
+ PRE as the first entry and the muxable PREs following.
+
+example:
+
+prg@21cc000 {
+ compatible = "fsl,imx6qp-prg";
+ reg = <0x021cc000 0x1000>;
+ clocks = <&clks IMX6QDL_CLK_PRG0_APB>,
+ <&clks IMX6QDL_CLK_PRG0_AXI>;
+ clock-names = "ipg", "axi";
+ fsl,pres = <&pre1>, <&pre2>, <&pre3>;
+};
+
Parallel display support
========================
- #address-cells: Should be <1>.
- #size-cells: Should be <0>.
- compatible: "rockchip,rk3288-mipi-dsi", "snps,dw-mipi-dsi".
+ "rockchip,rk3399-mipi-dsi", "snps,dw-mipi-dsi".
- reg: Represent the physical address range of the controller.
- interrupts: Represent the controller's interrupt to the CPU(s).
- clocks, clock-names: Phandles to the controller's pll reference
- clock(ref) and APB clock(pclk), as described in [1].
+ clock(ref) and APB clock(pclk). For RK3399, a phy config clock
+ (phy_cfg) is additional required. As described in [1].
- rockchip,grf: this soc should set GRF regs to mux vopl/vopb.
- ports: contain a port node with endpoint definitions as defined in [2].
For vopb,set the reg = <0> and set the reg = <1> for vopl.
+Optional properties:
+- power-domains: a phandle to mipi dsi power domain node.
+
[1] Documentation/devicetree/bindings/clock/clock-bindings.txt
[2] Documentation/devicetree/bindings/media/video-interfaces.txt
mcube mCube
meas Measurement Specialties
mediatek MediaTek Inc.
+megachips MegaChips
melexis Melexis N.V.
melfas MELFAS Inc.
memsic MEMSIC Inc.
--- /dev/null
+graph G {
+ Hello -- World
+}
To generate documentation, Sphinx (``sphinx-build``) must obviously be
installed. For prettier HTML output, the Read the Docs Sphinx theme
-(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is also
-needed. All of these are widely available and packaged in distributions.
+(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
+``XeLaTeX`` and ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
+All of these are widely available and packaged in distributions.
To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
Here are some specific guidelines for the kernel documentation:
-* Please don't go overboard with reStructuredText markup. Keep it simple.
+* Please don't go overboard with reStructuredText markup. Keep it
+ simple. For the most part the documentation should be plain text with
+ just enough consistency in formatting that it can be converted to
+ other formats.
+
+* Please keep the formatting changes minimal when converting existing
+ documentation to reStructuredText.
+
+* Also update the content, not just the formatting, when converting
+ documentation.
* Please stick to this order of heading adornments:
the order as encountered."), having the higher levels the same overall makes
it easier to follow the documents.
+* For inserting fixed width text blocks (for code examples, use case
+ examples, etc.), use ``::`` for anything that doesn't really benefit
+ from syntax highlighting, especially short snippets. Use
+ ``.. code-block:: <language>`` for longer code blocks that benefit
+ from highlighting.
+
the C domain
------------
* .. _`last row`:
- column 3
+
+
+Figures & Images
+================
+
+If you want to add an image, you should use the ``kernel-figure`` and
+``kernel-image`` directives. E.g. to insert a figure with a scalable
+image format use SVG (:ref:`svg_image_example`)::
+
+ .. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+.. _svg_image_example:
+
+.. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+The kernel figure (and image) directive support **DOT** formated files, see
+
+* DOT: http://graphviz.org/pdf/dotguide.pdf
+* Graphviz: http://www.graphviz.org/content/dot-language
+
+A simple example (:ref:`hello_dot_file`)::
+
+ .. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+.. _hello_dot_file:
+
+.. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the
+``kernel-render`` directives.::
+
+ .. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+How this will be rendered depends on the installed tools. If Graphviz is
+installed, you will see an vector image. If not the raw markup is inserted as
+*literal-block* (:ref:`hello_dot_render`).
+
+.. _hello_dot_render:
+
+.. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+The *render* directive has all the options known from the *figure* directive,
+plus option ``caption``. If ``caption`` has a value, a *figure* node is
+inserted. If not, a *image* node is inserted. A ``caption`` is also needed, if
+you want to refer it (:ref:`hello_svg_render`).
+
+Embedded **SVG**::
+
+ .. kernel-render:: SVG
+ :caption: Embedded **SVG** markup
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg" version="1.1" ...>
+ ...
+ </svg>
+
+.. _hello_svg_render:
+
+.. kernel-render:: SVG
+ :caption: Embedded **SVG** markup
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full" width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+ </svg>
--- /dev/null
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- originate: https://commons.wikimedia.org/wiki/File:Variable_Resistor.svg -->
+<svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full"
+ width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="0" y1="200" x2="700" y2="200" stroke="black" stroke-width="20px"/>
+ <rect x="100" y="100" width="500" height="200" fill="white" stroke="black" stroke-width="20px"/>
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+</svg>
.. kernel-doc:: drivers/gpu/drm/drm_drv.c
:doc: driver instance overview
-.. kernel-doc:: drivers/gpu/drm/drm_drv.c
- :export:
-
.. kernel-doc:: include/drm/drm_drv.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_drv.c
+ :export:
+
Driver Load
-----------
.. kernel-doc:: drivers/gpu/drm/drm_pci.c
:export:
-.. kernel-doc:: drivers/gpu/drm/drm_platform.c
- :export:
-
Open/Close, File Operations and IOCTLs
======================================
-Open and Close
---------------
-
-Open and close handlers. None of those methods are mandatory::
-
- int (*firstopen) (struct drm_device *);
- void (*lastclose) (struct drm_device *);
- int (*open) (struct drm_device *, struct drm_file *);
- void (*preclose) (struct drm_device *, struct drm_file *);
- void (*postclose) (struct drm_device *, struct drm_file *);
-
-The firstopen method is called by the DRM core for legacy UMS (User Mode
-Setting) drivers only when an application opens a device that has no
-other opened file handle. UMS drivers can implement it to acquire device
-resources. KMS drivers can't use the method and must acquire resources
-in the load method instead.
-
-Similarly the lastclose method is called when the last application
-holding a file handle opened on the device closes it, for both UMS and
-KMS drivers. Additionally, the method is also called at module unload
-time or, for hot-pluggable devices, when the device is unplugged. The
-firstopen and lastclose calls can thus be unbalanced.
-
-The open method is called every time the device is opened by an
-application. Drivers can allocate per-file private data in this method
-and store them in the struct :c:type:`struct drm_file
-<drm_file>` driver_priv field. Note that the open method is
-called before firstopen.
-
-The close operation is split into preclose and postclose methods.
-Drivers must stop and cleanup all per-file operations in the preclose
-method. For instance pending vertical blanking and page flip events must
-be cancelled. No per-file operation is allowed on the file handle after
-returning from the preclose method.
-
-Finally the postclose method is called as the last step of the close
-operation, right before calling the lastclose method if no other open
-file handle exists for the device. Drivers that have allocated per-file
-private data in the open method should free it here.
-
-The lastclose method should restore CRTC and plane properties to default
-value, so that a subsequent open of the device will not inherit state
-from the previous user. It can also be used to execute delayed power
-switching state changes, e.g. in conjunction with the :ref:`vga_switcheroo`
-infrastructure. Beyond that KMS drivers should not do any
-further cleanup. Only legacy UMS drivers might need to clean up device
-state so that the vga console or an independent fbdev driver could take
-over.
-
File Operations
---------------
-.. kernel-doc:: drivers/gpu/drm/drm_fops.c
+.. kernel-doc:: drivers/gpu/drm/drm_file.c
:doc: file operations
-.. kernel-doc:: drivers/gpu/drm/drm_fops.c
+.. kernel-doc:: include/drm/drm_file.h
+ :internal:
+
+.. kernel-doc:: drivers/gpu/drm/drm_file.c
:export:
IOCTLs
===========================================
.. kernel-doc:: include/drm/drm_modeset_helper_vtables.h
- :internal:
+ :doc: overview
.. kernel-doc:: include/drm/drm_modeset_helper_vtables.h
- :doc: overview
+ :internal:
+
+.. _drm_atomic_helper:
Atomic Modeset Helper Functions Reference
=========================================
Simple KMS Helper Reference
===========================
+.. kernel-doc:: drivers/gpu/drm/drm_simple_kms_helper.c
+ :doc: overview
+
.. kernel-doc:: include/drm/drm_simple_kms_helper.h
:internal:
.. kernel-doc:: drivers/gpu/drm/drm_simple_kms_helper.c
:export:
-.. kernel-doc:: drivers/gpu/drm/drm_simple_kms_helper.c
- :doc: overview
-
fbdev Helper Functions Reference
================================
.. kernel-doc:: drivers/gpu/drm/drm_fb_helper.c
:doc: fbdev helpers
-.. kernel-doc:: drivers/gpu/drm/drm_fb_helper.c
- :export:
-
.. kernel-doc:: include/drm/drm_fb_helper.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_fb_helper.c
+ :export:
+
Framebuffer CMA Helper Functions Reference
==========================================
.. kernel-doc:: drivers/gpu/drm/drm_fb_cma_helper.c
:export:
+.. _drm_bridges:
+
Bridges
=======
.. kernel-doc:: drivers/gpu/drm/drm_bridge.c
:export:
+.. _drm_panel_helper:
+
Panel Helper Reference
======================
+.. kernel-doc:: drivers/gpu/drm/drm_panel.c
+ :doc: drm panel
+
.. kernel-doc:: include/drm/drm_panel.h
:internal:
.. kernel-doc:: drivers/gpu/drm/drm_panel.c
:export:
-.. kernel-doc:: drivers/gpu/drm/drm_panel.c
- :doc: drm panel
-
Display Port Helper Functions Reference
=======================================
.. kernel-doc:: drivers/gpu/drm/drm_edid.c
:export:
+SCDC Helper Functions Reference
+===============================
+
+.. kernel-doc:: drivers/gpu/drm/drm_scdc_helper.c
+ :doc: scdc helpers
+
+.. kernel-doc:: include/drm/drm_scdc_helper.h
+ :internal:
+
+.. kernel-doc:: drivers/gpu/drm/drm_scdc_helper.c
+ :export:
+
Rectangle Utilities Reference
=============================
- struct drm_mode_config_funcs \*funcs;
Mode setting functions.
-Mode Configuration
+Overview
+========
+
+.. kernel-render:: DOT
+ :alt: KMS Display Pipeline
+ :caption: KMS Display Pipeline Overview
+
+ digraph "KMS" {
+ node [shape=box]
+
+ subgraph cluster_static {
+ style=dashed
+ label="Static Objects"
+
+ node [bgcolor=grey style=filled]
+ "drm_plane A" -> "drm_crtc"
+ "drm_plane B" -> "drm_crtc"
+ "drm_crtc" -> "drm_encoder A"
+ "drm_crtc" -> "drm_encoder B"
+ }
+
+ subgraph cluster_user_created {
+ style=dashed
+ label="Userspace-Created"
+
+ node [shape=oval]
+ "drm_framebuffer 1" -> "drm_plane A"
+ "drm_framebuffer 2" -> "drm_plane B"
+ }
+
+ subgraph cluster_connector {
+ style=dashed
+ label="Hotpluggable"
+
+ "drm_encoder A" -> "drm_connector A"
+ "drm_encoder B" -> "drm_connector B"
+ }
+ }
+
+The basic object structure KMS presents to userspace is fairly simple.
+Framebuffers (represented by :c:type:`struct drm_framebuffer <drm_framebuffer>`,
+see `Frame Buffer Abstraction`_) feed into planes. One or more (or even no)
+planes feed their pixel data into a CRTC (represented by :c:type:`struct
+drm_crtc <drm_crtc>`, see `CRTC Abstraction`_) for blending. The precise
+blending step is explained in more detail in `Plane Composition Properties`_ and
+related chapters.
+
+For the output routing the first step is encoders (represented by
+:c:type:`struct drm_encoder <drm_encoder>`, see `Encoder Abstraction`_). Those
+are really just internal artifacts of the helper libraries used to implement KMS
+drivers. Besides that they make it unecessarily more complicated for userspace
+to figure out which connections between a CRTC and a connector are possible, and
+what kind of cloning is supported, they serve no purpose in the userspace API.
+Unfortunately encoders have been exposed to userspace, hence can't remove them
+at this point. Futhermore the exposed restrictions are often wrongly set by
+drivers, and in many cases not powerful enough to express the real restrictions.
+A CRTC can be connected to multiple encoders, and for an active CRTC there must
+be at least one encoder.
+
+The final, and real, endpoint in the display chain is the connector (represented
+by :c:type:`struct drm_connector <drm_connector>`, see `Connector
+Abstraction`_). Connectors can have different possible encoders, but the kernel
+driver selects which encoder to use for each connector. The use case is DVI,
+which could switch between an analog and a digital encoder. Encoders can also
+drive multiple different connectors. There is exactly one active connector for
+every active encoder.
+
+Internally the output pipeline is a bit more complex and matches today's
+hardware more closely:
+
+.. kernel-render:: DOT
+ :alt: KMS Output Pipeline
+ :caption: KMS Output Pipeline
+
+ digraph "Output Pipeline" {
+ node [shape=box]
+
+ subgraph {
+ "drm_crtc" [bgcolor=grey style=filled]
+ }
+
+ subgraph cluster_internal {
+ style=dashed
+ label="Internal Pipeline"
+ {
+ node [bgcolor=grey style=filled]
+ "drm_encoder A";
+ "drm_encoder B";
+ "drm_encoder C";
+ }
+
+ {
+ node [bgcolor=grey style=filled]
+ "drm_encoder B" -> "drm_bridge B"
+ "drm_encoder C" -> "drm_bridge C1"
+ "drm_bridge C1" -> "drm_bridge C2";
+ }
+ }
+
+ "drm_crtc" -> "drm_encoder A"
+ "drm_crtc" -> "drm_encoder B"
+ "drm_crtc" -> "drm_encoder C"
+
+
+ subgraph cluster_output {
+ style=dashed
+ label="Outputs"
+
+ "drm_encoder A" -> "drm_connector A";
+ "drm_bridge B" -> "drm_connector B";
+ "drm_bridge C2" -> "drm_connector C";
+
+ "drm_panel"
+ }
+ }
+
+Internally two additional helper objects come into play. First, to be able to
+share code for encoders (sometimes on the same SoC, sometimes off-chip) one or
+more :ref:`drm_bridges` (represented by :c:type:`struct drm_bridge
+<drm_bridge>`) can be linked to an encoder. This link is static and cannot be
+changed, which means the cross-bar (if there is any) needs to be mapped between
+the CRTC and any encoders. Often for drivers with bridges there's no code left
+at the encoder level. Atomic drivers can leave out all the encoder callbacks to
+essentially only leave a dummy routing object behind, which is needed for
+backwards compatibility since encoders are exposed to userspace.
+
+The second object is for panels, represented by :c:type:`struct drm_panel
+<drm_panel>`, see :ref:`drm_panel_helper`. Panels do not have a fixed binding
+point, but are generally linked to the driver private structure that embeds
+:c:type:`struct drm_connector <drm_connector>`.
+
+Note that currently the bridge chaining and interactions with connectors and
+panels are still in-flux and not really fully sorted out yet.
KMS Core Structures and Functions
=================================
-.. kernel-doc:: drivers/gpu/drm/drm_mode_config.c
- :export:
-
.. kernel-doc:: include/drm/drm_mode_config.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_mode_config.c
+ :export:
+
Modeset Base Object Abstraction
===============================
+.. kernel-render:: DOT
+ :alt: Mode Objects and Properties
+ :caption: Mode Objects and Properties
+
+ digraph {
+ node [shape=box]
+
+ "drm_property A" -> "drm_mode_object A"
+ "drm_property A" -> "drm_mode_object B"
+ "drm_property B" -> "drm_mode_object A"
+ }
+
+The base structure for all KMS objects is :c:type:`struct drm_mode_object
+<drm_mode_object>`. One of the base services it provides is tracking properties,
+which are especially important for the atomic IOCTL (see `Atomic Mode
+Setting`_). The somewhat surprising part here is that properties are not
+directly instantiated on each object, but free-standing mode objects themselves,
+represented by :c:type:`struct drm_property <drm_property>`, which only specify
+the type and value range of a property. Any given property can be attached
+multiple times to different objects using :c:func:`drm_object_attach_property()
+<drm_object_attach_property>`.
+
.. kernel-doc:: include/drm/drm_mode_object.h
:internal:
.. kernel-doc:: drivers/gpu/drm/drm_mode_object.c
:export:
-Atomic Mode Setting Function Reference
-======================================
+Atomic Mode Setting
+===================
-.. kernel-doc:: drivers/gpu/drm/drm_atomic.c
- :export:
+
+.. kernel-render:: DOT
+ :alt: Mode Objects and Properties
+ :caption: Mode Objects and Properties
+
+ digraph {
+ node [shape=box]
+
+ subgraph cluster_state {
+ style=dashed
+ label="Free-standing state"
+
+ "drm_atomic_state" -> "duplicated drm_plane_state A"
+ "drm_atomic_state" -> "duplicated drm_plane_state B"
+ "drm_atomic_state" -> "duplicated drm_crtc_state"
+ "drm_atomic_state" -> "duplicated drm_connector_state"
+ "drm_atomic_state" -> "duplicated driver private state"
+ }
+
+ subgraph cluster_current {
+ style=dashed
+ label="Current state"
+
+ "drm_device" -> "drm_plane A"
+ "drm_device" -> "drm_plane B"
+ "drm_device" -> "drm_crtc"
+ "drm_device" -> "drm_connector"
+ "drm_device" -> "driver private object"
+
+ "drm_plane A" -> "drm_plane_state A"
+ "drm_plane B" -> "drm_plane_state B"
+ "drm_crtc" -> "drm_crtc_state"
+ "drm_connector" -> "drm_connector_state"
+ "driver private object" -> "driver private state"
+ }
+
+ "drm_atomic_state" -> "drm_device" [label="atomic_commit"]
+ "duplicated drm_plane_state A" -> "drm_device"[style=invis]
+ }
+
+Atomic provides transactional modeset (including planes) updates, but a
+bit differently from the usual transactional approach of try-commit and
+rollback:
+
+- Firstly, no hardware changes are allowed when the commit would fail. This
+ allows us to implement the DRM_MODE_ATOMIC_TEST_ONLY mode, which allows
+ userspace to explore whether certain configurations would work or not.
+
+- This would still allow setting and rollback of just the software state,
+ simplifying conversion of existing drivers. But auditing drivers for
+ correctness of the atomic_check code becomes really hard with that: Rolling
+ back changes in data structures all over the place is hard to get right.
+
+- Lastly, for backwards compatibility and to support all use-cases, atomic
+ updates need to be incremental and be able to execute in parallel. Hardware
+ doesn't always allow it, but where possible plane updates on different CRTCs
+ should not interfere, and not get stalled due to output routing changing on
+ different CRTCs.
+
+Taken all together there's two consequences for the atomic design:
+
+- The overall state is split up into per-object state structures:
+ :c:type:`struct drm_plane_state <drm_plane_state>` for planes, :c:type:`struct
+ drm_crtc_state <drm_crtc_state>` for CRTCs and :c:type:`struct
+ drm_connector_state <drm_connector_state>` for connectors. These are the only
+ objects with userspace-visible and settable state. For internal state drivers
+ can subclass these structures through embeddeding, or add entirely new state
+ structures for their globally shared hardware functions.
+
+- An atomic update is assembled and validated as an entirely free-standing pile
+ of structures within the :c:type:`drm_atomic_state <drm_atomic_state>`
+ container. Again drivers can subclass that container for their own state
+ structure tracking needs. Only when a state is committed is it applied to the
+ driver and modeset objects. This way rolling back an update boils down to
+ releasing memory and unreferencing objects like framebuffers.
+
+Read on in this chapter, and also in :ref:`drm_atomic_helper` for more detailed
+coverage of specific topics.
+
+Atomic Mode Setting Function Reference
+--------------------------------------
.. kernel-doc:: include/drm/drm_atomic.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_atomic.c
+ :export:
+
CRTC Abstraction
================
Frame Buffer Functions Reference
--------------------------------
-.. kernel-doc:: drivers/gpu/drm/drm_framebuffer.c
- :export:
-
.. kernel-doc:: include/drm/drm_framebuffer.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_framebuffer.c
+ :export:
+
DRM Format Handling
===================
Vertical Blanking and Interrupt Handling Functions Reference
------------------------------------------------------------
-.. kernel-doc:: drivers/gpu/drm/drm_irq.c
- :export:
-
.. kernel-doc:: include/drm/drm_irq.h
:internal:
+
+.. kernel-doc:: drivers/gpu/drm/drm_irq.c
+ :export:
--------------------
All GEM objects are reference-counted by the GEM core. References can be
-acquired and release by :c:func:`calling
-drm_gem_object_reference()` and
-:c:func:`drm_gem_object_unreference()` respectively. The caller
-must hold the :c:type:`struct drm_device <drm_device>`
-struct_mutex lock when calling
-:c:func:`drm_gem_object_reference()`. As a convenience, GEM
-provides :c:func:`drm_gem_object_unreference_unlocked()`
-functions that can be called without holding the lock.
+acquired and release by :c:func:`calling drm_gem_object_get()` and
+:c:func:`drm_gem_object_put()` respectively. The caller must hold the
+:c:type:`struct drm_device <drm_device>` struct_mutex lock when calling
+:c:func:`drm_gem_object_get()`. As a convenience, GEM provides
+:c:func:`drm_gem_object_put_unlocked()` functions that can be called without
+holding the lock.
When the last reference to a GEM object is released the GEM core calls
the :c:type:`struct drm_driver <drm_driver>` gem_free_object
GEM Function Reference
----------------------
-.. kernel-doc:: drivers/gpu/drm/drm_gem.c
- :export:
-
.. kernel-doc:: include/drm/drm_gem.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_gem.c
+ :export:
+
GEM CMA Helper Functions Reference
----------------------------------
.. kernel-doc:: drivers/gpu/drm/drm_gem_cma_helper.c
:doc: cma helpers
-.. kernel-doc:: drivers/gpu/drm/drm_gem_cma_helper.c
- :export:
-
.. kernel-doc:: include/drm/drm_gem_cma_helper.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_gem_cma_helper.c
+ :export:
+
VMA Offset Manager
==================
.. kernel-doc:: drivers/gpu/drm/drm_vma_manager.c
:doc: vma offset manager
-.. kernel-doc:: drivers/gpu/drm/drm_vma_manager.c
- :export:
-
.. kernel-doc:: include/drm/drm_vma_manager.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_vma_manager.c
+ :export:
+
PRIME Buffer Sharing
====================
PRIME Function References
-------------------------
+.. kernel-doc:: include/drm/drm_prime.h
+ :internal:
+
.. kernel-doc:: drivers/gpu/drm/drm_prime.c
:export:
DRM MM Range Allocator Function References
------------------------------------------
-.. kernel-doc:: drivers/gpu/drm/drm_mm.c
- :export:
-
.. kernel-doc:: include/drm/drm_mm.h
:internal:
+.. kernel-doc:: drivers/gpu/drm/drm_mm.c
+ :export:
+
DRM Cache Handling
==================
:doc: getunique and setversion story
+.. _drm_primary_node:
+
Primary Nodes, DRM Master and Authentication
============================================
for the same thing co-existing. If we add a few more complete mistakes into the
mix every year it would be entirely unmanageable.
+.. _drm_render_node:
+
Render nodes
============
.. kernel-doc:: drivers/gpu/drm/i915/intel_vbt_defs.h
:internal:
+Display clocks
+--------------
+
+.. kernel-doc:: drivers/gpu/drm/i915/intel_cdclk.c
+ :doc: CDCLK / RAWCLK
+
+.. kernel-doc:: drivers/gpu/drm/i915/intel_cdclk.c
+ :internal:
+
Display PLLs
------------
drm-uapi
i915
tinydrm
+ vc4
vga-switcheroo
vgaarbiter
+ todo
.. only:: subproject and html
and "FIXME" where the interface could be cleaned up.
Also read the :ref:`guidelines for the kernel documentation at large <doc_guide>`.
+
+Getting Started
+===============
+
+Developers interested in helping out with the DRM subsystem are very welcome.
+Often people will resort to sending in patches for various issues reported by
+checkpatch or sparse. We welcome such contributions.
+
+Anyone looking to kick it up a notch can find a list of janitorial tasks on
+the :ref:`TODO list <todo>`.
--- /dev/null
+.. _todo:
+
+=========
+TODO list
+=========
+
+This section contains a list of smaller janitorial tasks in the kernel DRM
+graphics subsystem useful as newbie projects. Or for slow rainy days.
+
+Subsystem-wide refactorings
+===========================
+
+De-midlayer drivers
+-------------------
+
+With the recent ``drm_bus`` cleanup patches for 3.17 it is no longer required
+to have a ``drm_bus`` structure set up. Drivers can directly set up the
+``drm_device`` structure instead of relying on bus methods in ``drm_usb.c``
+and ``drm_platform.c``. The goal is to get rid of the driver's ``->load`` /
+``->unload`` callbacks and open-code the load/unload sequence properly, using
+the new two-stage ``drm_device`` setup/teardown.
+
+Once all existing drivers are converted we can also remove those bus support
+files for USB and platform devices.
+
+All you need is a GPU for a non-converted driver (currently almost all of
+them, but also all the virtual ones used by KVM, so everyone qualifies).
+
+Contact: Daniel Vetter, Thierry Reding, respective driver maintainers
+
+Switch from reference/unreference to get/put
+--------------------------------------------
+
+For some reason DRM core uses ``reference``/``unreference`` suffixes for
+refcounting functions, but kernel uses ``get``/``put`` (e.g.
+``kref_get``/``put()``). It would be good to switch over for consistency, and
+it's shorter. Needs to be done in 3 steps for each pair of functions:
+
+* Create new ``get``/``put`` functions, define the old names as compatibility
+ wrappers
+* Switch over each file/driver using a cocci-generated spatch.
+* Once all users of the old names are gone, remove them.
+
+This way drivers/patches in the progress of getting merged won't break.
+
+Contact: Daniel Vetter
+
+Convert existing KMS drivers to atomic modesetting
+--------------------------------------------------
+
+3.19 has the atomic modeset interfaces and helpers, so drivers can now be
+converted over. Modern compositors like Wayland or Surfaceflinger on Android
+really want an atomic modeset interface, so this is all about the bright
+future.
+
+There is a conversion guide for atomic and all you need is a GPU for a
+non-converted driver (again virtual HW drivers for KVM are still all
+suitable).
+
+As part of this drivers also need to convert to universal plane (which means
+exposing primary & cursor as proper plane objects). But that's much easier to
+do by directly using the new atomic helper driver callbacks.
+
+Contact: Daniel Vetter, respective driver maintainers
+
+Clean up the clipped coordination confusion around planes
+---------------------------------------------------------
+
+We have a helper to get this right with drm_plane_helper_check_update(), but
+it's not consistently used. This should be fixed, preferrably in the atomic
+helpers (and drivers then moved over to clipped coordinates). Probably the
+helper should also be moved from drm_plane_helper.c to the atomic helpers, to
+avoid confusion - the other helpers in that file are all deprecated legacy
+helpers.
+
+Contact: Ville Syrjälä, Daniel Vetter, driver maintainers
+
+Implement deferred fbdev setup in the helper
+--------------------------------------------
+
+Many (especially embedded drivers) want to delay fbdev setup until there's a
+real screen plugged in. This is to avoid the dreaded fallback to the low-res
+fbdev default. Many drivers have a hacked-up (and often broken) version of this,
+better to do it once in the shared helpers. Thierry has a patch series, but that
+one needs to be rebased and final polish applied.
+
+Contact: Thierry Reding, Daniel Vetter, driver maintainers
+
+Convert early atomic drivers to async commit helpers
+----------------------------------------------------
+
+For the first year the atomic modeset helpers didn't support asynchronous /
+nonblocking commits, and every driver had to hand-roll them. This is fixed
+now, but there's still a pile of existing drivers that easily could be
+converted over to the new infrastructure.
+
+One issue with the helpers is that they require that drivers handle completion
+events for atomic commits correctly. But fixing these bugs is good anyway.
+
+Contact: Daniel Vetter, respective driver maintainers
+
+Fallout from atomic KMS
+-----------------------
+
+``drm_atomic_helper.c`` provides a batch of functions which implement legacy
+IOCTLs on top of the new atomic driver interface. Which is really nice for
+gradual conversion of drivers, but unfortunately the semantic mismatches are
+a bit too severe. So there's some follow-up work to adjust the function
+interfaces to fix these issues:
+
+* atomic needs the lock acquire context. At the moment that's passed around
+ implicitly with some horrible hacks, and it's also allocate with
+ ``GFP_NOFAIL`` behind the scenes. All legacy paths need to start allocating
+ the acquire context explicitly on stack and then also pass it down into
+ drivers explicitly so that the legacy-on-atomic functions can use them.
+
+* A bunch of the vtable hooks are now in the wrong place: DRM has a split
+ between core vfunc tables (named ``drm_foo_funcs``), which are used to
+ implement the userspace ABI. And then there's the optional hooks for the
+ helper libraries (name ``drm_foo_helper_funcs``), which are purely for
+ internal use. Some of these hooks should be move from ``_funcs`` to
+ ``_helper_funcs`` since they are not part of the core ABI. There's a
+ ``FIXME`` comment in the kerneldoc for each such case in ``drm_crtc.h``.
+
+* There's a new helper ``drm_atomic_helper_best_encoder()`` which could be
+ used by all atomic drivers which don't select the encoder for a given
+ connector at runtime. That's almost all of them, and would allow us to get
+ rid of a lot of ``best_encoder`` boilerplate in drivers.
+
+Contact: Daniel Vetter
+
+Get rid of dev->struct_mutex from GEM drivers
+---------------------------------------------
+
+``dev->struct_mutex`` is the Big DRM Lock from legacy days and infested
+everything. Nowadays in modern drivers the only bit where it's mandatory is
+serializing GEM buffer object destruction. Which unfortunately means drivers
+have to keep track of that lock and either call ``unreference`` or
+``unreference_locked`` depending upon context.
+
+Core GEM doesn't have a need for ``struct_mutex`` any more since kernel 4.8,
+and there's a ``gem_free_object_unlocked`` callback for any drivers which are
+entirely ``struct_mutex`` free.
+
+For drivers that need ``struct_mutex`` it should be replaced with a driver-
+private lock. The tricky part is the BO free functions, since those can't
+reliably take that lock any more. Instead state needs to be protected with
+suitable subordinate locks or some cleanup work pushed to a worker thread. For
+performance-critical drivers it might also be better to go with a more
+fine-grained per-buffer object and per-context lockings scheme. Currently the
+following drivers still use ``struct_mutex``: ``msm``, ``omapdrm`` and
+``udl``.
+
+Contact: Daniel Vetter
+
+Switch to drm_connector_list_iter for any connector_list walking
+----------------------------------------------------------------
+
+Connectors can be hotplugged, and we now have a special list of helpers to walk
+the connector_list in a race-free fashion, without incurring deadlocks on
+mutexes and other fun stuff.
+
+Unfortunately most drivers are not converted yet. At least all those supporting
+DP MST hotplug should be converted, since for those drivers the difference
+matters. See drm_for_each_connector_iter() vs. drm_for_each_connector().
+
+Contact: Daniel Vetter
+
+Core refactorings
+=================
+
+Use new IDR deletion interface to clean up drm_gem_handle_delete()
+------------------------------------------------------------------
+
+See the "This is gross" comment -- apparently the IDR system now can return an
+error code instead of oopsing.
+
+Clean up the DRM header mess
+----------------------------
+
+Currently the DRM subsystem has only one global header, ``drmP.h``. This is
+used both for functions exported to helper libraries and drivers and functions
+only used internally in the ``drm.ko`` module. The goal would be to move all
+header declarations not needed outside of ``drm.ko`` into
+``drivers/gpu/drm/drm_*_internal.h`` header files. ``EXPORT_SYMBOL`` also
+needs to be dropped for these functions.
+
+This would nicely tie in with the below task to create kerneldoc after the API
+is cleaned up. Or with the "hide legacy cruft better" task.
+
+Note that this is well in progress, but ``drmP.h`` is still huge. The updated
+plan is to switch to per-file driver API headers, which will also structure
+the kerneldoc better. This should also allow more fine-grained ``#include``
+directives.
+
+Contact: Daniel Vetter
+
+Add missing kerneldoc for exported functions
+--------------------------------------------
+
+The DRM reference documentation is still lacking kerneldoc in a few areas. The
+task would be to clean up interfaces like moving functions around between
+files to better group them and improving the interfaces like dropping return
+values for functions that never fail. Then write kerneldoc for all exported
+functions and an overview section and integrate it all into the drm DocBook.
+
+See https://dri.freedesktop.org/docs/drm/ for what's there already.
+
+Contact: Daniel Vetter
+
+Hide legacy cruft better
+------------------------
+
+Way back DRM supported only drivers which shadow-attached to PCI devices with
+userspace or fbdev drivers setting up outputs. Modern DRM drivers take charge
+of the entire device, you can spot them with the DRIVER_MODESET flag.
+
+Unfortunately there's still large piles of legacy code around which needs to
+be hidden so that driver writers don't accidentally end up using it. And to
+prevent security issues in those legacy IOCTLs from being exploited on modern
+drivers. This has multiple possible subtasks:
+
+* Make sure legacy IOCTLs can't be used on modern drivers.
+* Extract support code for legacy features into a ``drm-legacy.ko`` kernel
+ module and compile it only when one of the legacy drivers is enabled.
+* Extract legacy functions into their own headers and remove it that from the
+ monolithic ``drmP.h`` header.
+* Remove any lingering cruft from the OS abstraction layer from modern
+ drivers.
+
+This is mostly done, the only thing left is to split up ``drm_irq.c`` into
+legacy cruft and the parts needed by modern KMS drivers.
+
+Contact: Daniel Vetter
+
+Make panic handling work
+------------------------
+
+This is a really varied tasks with lots of little bits and pieces:
+
+* The panic path can't be tested currently, leading to constant breaking. The
+ main issue here is that panics can be triggered from hardirq contexts and
+ hence all panic related callback can run in hardirq context. It would be
+ awesome if we could test at least the fbdev helper code and driver code by
+ e.g. trigger calls through drm debugfs files. hardirq context could be
+ achieved by using an IPI to the local processor.
+
+* There's a massive confusion of different panic handlers. DRM fbdev emulation
+ helpers have one, but on top of that the fbcon code itself also has one. We
+ need to make sure that they stop fighting over each another.
+
+* ``drm_can_sleep()`` is a mess. It hides real bugs in normal operations and
+ isn't a full solution for panic paths. We need to make sure that it only
+ returns true if there's a panic going on for real, and fix up all the
+ fallout.
+
+* The panic handler must never sleep, which also means it can't ever
+ ``mutex_lock()``. Also it can't grab any other lock unconditionally, not
+ even spinlocks (because NMI and hardirq can panic too). We need to either
+ make sure to not call such paths, or trylock everything. Really tricky.
+
+* For the above locking troubles reasons it's pretty much impossible to
+ attempt a synchronous modeset from panic handlers. The only thing we could
+ try to achive is an atomic ``set_base`` of the primary plane, and hope that
+ it shows up. Everything else probably needs to be delayed to some worker or
+ something else which happens later on. Otherwise it just kills the box
+ harder, prevent the panic from going out on e.g. netconsole.
+
+* There's also proposal for a simplied DRM console instead of the full-blown
+ fbcon and DRM fbdev emulation. Any kind of panic handling tricks should
+ obviously work for both console, in case we ever get kmslog merged.
+
+Contact: Daniel Vetter
+
+Better Testing
+==============
+
+Enable trinity for DRM
+----------------------
+
+And fix up the fallout. Should be really interesting ...
+
+Make KMS tests in i-g-t generic
+-------------------------------
+
+The i915 driver team maintains an extensive testsuite for the i915 DRM driver,
+including tons of testcases for corner-cases in the modesetting API. It would
+be awesome if those tests (at least the ones not relying on Intel-specific GEM
+features) could be made to run on any KMS driver.
+
+Basic work to run i-g-t tests on non-i915 is done, what's now missing is mass-
+converting things over. For modeset tests we also first need a bit of
+infrastructure to use dumb buffers for untiled buffers, to be able to run all
+the non-i915 specific modeset tests.
+
+Contact: Daniel Vetter
+
+Create a virtual KMS driver for testing (vkms)
+----------------------------------------------
+
+With all the latest helpers it should be fairly simple to create a virtual KMS
+driver useful for testing, or for running X or similar on headless machines
+(to be able to still use the GPU). This would be similar to vgem, but aimed at
+the modeset side.
+
+Once the basics are there there's tons of possibilities to extend it.
+
+Contact: Daniel Vetter
+
+Driver Specific
+===============
+
+Outside DRM
+===========
+
+Better kerneldoc
+----------------
+
+This is pretty much done, but there's some advanced topics:
+
+Come up with a way to hyperlink to struct members. Currently you can hyperlink
+to the struct using ``#struct_name``, but not to a member within. Would need
+buy-in from kerneldoc maintainers, and the big question is how to make it work
+without totally unsightly
+``drm_foo_bar_really_long_structure->even_longer_memeber`` all over the text
+which breaks text flow.
+
+Figure out how to integrate the asciidoc support for ascii-diagrams. We have a
+few of those (e.g. to describe mode timings), and asciidoc supports converting
+some ascii-art dialect into pngs. Would be really pretty to make that work.
+
+Contact: Daniel Vetter, Jani Nikula
+
+Jani is working on this already, hopefully lands in 4.8.
--- /dev/null
+=====================================
+ drm/vc4 Broadcom VC4 Graphics Driver
+=====================================
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_drv.c
+ :doc: Broadcom VC4 Graphics Driver
+
+Display Hardware Handling
+=========================
+
+This section covers everything related to the display hardware including
+the mode setting infrastructure, plane, sprite and cursor handling and
+display, output probing and related topics.
+
+Pixel Valve (DRM CRTC)
+----------------------
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_crtc.c
+ :doc: VC4 CRTC module
+
+HVS
+---
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_hvs.c
+ :doc: VC4 HVS module.
+
+HVS planes
+----------
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_plane.c
+ :doc: VC4 plane module
+
+HDMI encoder
+------------
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_hdmi.c
+ :doc: VC4 Falcon HDMI module
+
+DSI encoder
+-----------
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_dsi.c
+ :doc: VC4 DSI0/DSI1 module
+
+DPI encoder
+-----------
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_dpi.c
+ :doc: VC4 DPI module
+
+VEC (Composite TV out) encoder
+------------------------------
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_vec.c
+ :doc: VC4 SDTV module
+
+Memory Management and 3D Command Submission
+===========================================
+
+This section covers the GEM implementation in the vc4 driver.
+
+GPU buffer object (BO) management
+---------------------------------
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_bo.c
+ :doc: VC4 GEM BO management support
+
+V3D binner command list (BCL) validation
+----------------------------------------
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_validate.c
+ :doc: Command list validator for VC4.
+
+V3D render command list (RCL) generation
+----------------------------------------
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_render_cl.c
+ :doc: Render command list generation
+
+Shader validator for VC4
+---------------------------
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_validate_shaders.c
+ :doc: Shader validator for VC4.
+
+V3D Interrupts
+--------------
+
+.. kernel-doc:: drivers/gpu/drm/vc4/vc4_irq.c
+ :doc: Interrupt management for the V3D engine
non-dt boot, it should contain the label of the PHY. The two
devm_phy_get associates the device with the PHY using devres on
successful PHY get. On driver detach, release function is invoked on
-the the devres data and devres data is freed. phy_optional_get and
+the devres data and devres data is freed. phy_optional_get and
devm_phy_optional_get should be used when the phy is optional. These
two functions will never return -ENODEV, but instead returns NULL when
the phy cannot be found.Some generic drivers, such as ehci, may use multiple
.. note::
Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX``
- version 3.14159265. Depending on the distribution, you may also need
- to install a series of ``texlive`` packages that provide the minimal
- set of functionalities required for ``XeLaTex`` to work.
+ version 3.14159265. Depending on the distribution, you may also need to
+ install a series of ``texlive`` packages that provide the minimal set of
+ functionalities required for ``XeLaTex`` to work. For PDF output you'll also
+ need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
Other tools
-----------
--- /dev/null
+# -*- coding: utf-8; mode: python -*-
+# pylint: disable=C0103, R0903, R0912, R0915
+u"""
+ scalable figure and image handling
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ Sphinx extension which implements scalable image handling.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :license: GPL Version 2, June 1991 see Linux/COPYING for details.
+
+ The build for image formats depend on image's source format and output's
+ destination format. This extension implement methods to simplify image
+ handling from the author's POV. Directives like ``kernel-figure`` implement
+ methods *to* always get the best output-format even if some tools are not
+ installed. For more details take a look at ``convert_image(...)`` which is
+ the core of all conversions.
+
+ * ``.. kernel-image``: for image handling / a ``.. image::`` replacement
+
+ * ``.. kernel-figure``: for figure handling / a ``.. figure::`` replacement
+
+ * ``.. kernel-render``: for render markup / a concept to embed *render*
+ markups (or languages). Supported markups (see ``RENDER_MARKUP_EXT``)
+
+ - ``DOT``: render embedded Graphviz's **DOC**
+ - ``SVG``: render embedded Scalable Vector Graphics (**SVG**)
+ - ... *developable*
+
+ Used tools:
+
+ * ``dot(1)``: Graphviz (http://www.graphviz.org). If Graphviz is not
+ available, the DOT language is inserted as literal-block.
+
+ * SVG to PDF: To generate PDF, you need at least one of this tools:
+
+ - ``convert(1)``: ImageMagick (https://www.imagemagick.org)
+
+ List of customizations:
+
+ * generate PDF from SVG / used by PDF (LaTeX) builder
+
+ * generate SVG (html-builder) and PDF (latex-builder) from DOT files.
+ DOT: see http://www.graphviz.org/content/dot-language
+
+ """
+
+import os
+from os import path
+import subprocess
+from hashlib import sha1
+import sys
+
+from docutils import nodes
+from docutils.statemachine import ViewList
+from docutils.parsers.rst import directives
+from docutils.parsers.rst.directives import images
+import sphinx
+
+from sphinx.util.nodes import clean_astext
+from six import iteritems
+
+PY3 = sys.version_info[0] == 3
+
+if PY3:
+ _unicode = str
+else:
+ _unicode = unicode
+
+# Get Sphinx version
+major, minor, patch = sphinx.version_info[:3]
+if major == 1 and minor > 3:
+ # patches.Figure only landed in Sphinx 1.4
+ from sphinx.directives.patches import Figure # pylint: disable=C0413
+else:
+ Figure = images.Figure
+
+__version__ = '1.0.0'
+
+# simple helper
+# -------------
+
+def which(cmd):
+ """Searches the ``cmd`` in the ``PATH`` enviroment.
+
+ This *which* searches the PATH for executable ``cmd`` . First match is
+ returned, if nothing is found, ``None` is returned.
+ """
+ envpath = os.environ.get('PATH', None) or os.defpath
+ for folder in envpath.split(os.pathsep):
+ fname = folder + os.sep + cmd
+ if path.isfile(fname):
+ return fname
+
+def mkdir(folder, mode=0o775):
+ if not path.isdir(folder):
+ os.makedirs(folder, mode)
+
+def file2literal(fname):
+ with open(fname, "r") as src:
+ data = src.read()
+ node = nodes.literal_block(data, data)
+ return node
+
+def isNewer(path1, path2):
+ """Returns True if ``path1`` is newer than ``path2``
+
+ If ``path1`` exists and is newer than ``path2`` the function returns
+ ``True`` is returned otherwise ``False``
+ """
+ return (path.exists(path1)
+ and os.stat(path1).st_ctime > os.stat(path2).st_ctime)
+
+def pass_handle(self, node): # pylint: disable=W0613
+ pass
+
+# setup conversion tools and sphinx extension
+# -------------------------------------------
+
+# Graphviz's dot(1) support
+dot_cmd = None
+
+# ImageMagick' convert(1) support
+convert_cmd = None
+
+
+def setup(app):
+ # check toolchain first
+ app.connect('builder-inited', setupTools)
+
+ # image handling
+ app.add_directive("kernel-image", KernelImage)
+ app.add_node(kernel_image,
+ html = (visit_kernel_image, pass_handle),
+ latex = (visit_kernel_image, pass_handle),
+ texinfo = (visit_kernel_image, pass_handle),
+ text = (visit_kernel_image, pass_handle),
+ man = (visit_kernel_image, pass_handle), )
+
+ # figure handling
+ app.add_directive("kernel-figure", KernelFigure)
+ app.add_node(kernel_figure,
+ html = (visit_kernel_figure, pass_handle),
+ latex = (visit_kernel_figure, pass_handle),
+ texinfo = (visit_kernel_figure, pass_handle),
+ text = (visit_kernel_figure, pass_handle),
+ man = (visit_kernel_figure, pass_handle), )
+
+ # render handling
+ app.add_directive('kernel-render', KernelRender)
+ app.add_node(kernel_render,
+ html = (visit_kernel_render, pass_handle),
+ latex = (visit_kernel_render, pass_handle),
+ texinfo = (visit_kernel_render, pass_handle),
+ text = (visit_kernel_render, pass_handle),
+ man = (visit_kernel_render, pass_handle), )
+
+ app.connect('doctree-read', add_kernel_figure_to_std_domain)
+
+ return dict(
+ version = __version__,
+ parallel_read_safe = True,
+ parallel_write_safe = True
+ )
+
+
+def setupTools(app):
+ u"""
+ Check available build tools and log some *verbose* messages.
+
+ This function is called once, when the builder is initiated.
+ """
+ global dot_cmd, convert_cmd # pylint: disable=W0603
+ app.verbose("kfigure: check installed tools ...")
+
+ dot_cmd = which('dot')
+ convert_cmd = which('convert')
+
+ if dot_cmd:
+ app.verbose("use dot(1) from: " + dot_cmd)
+ else:
+ app.warn("dot(1) not found, for better output quality install "
+ "graphviz from http://www.graphviz.org")
+ if convert_cmd:
+ app.verbose("use convert(1) from: " + convert_cmd)
+ else:
+ app.warn(
+ "convert(1) not found, for SVG to PDF conversion install "
+ "ImageMagick (https://www.imagemagick.org)")
+
+
+# integrate conversion tools
+# --------------------------
+
+RENDER_MARKUP_EXT = {
+ # The '.ext' must be handled by convert_image(..) function's *in_ext* input.
+ # <name> : <.ext>
+ 'DOT' : '.dot',
+ 'SVG' : '.svg'
+}
+
+def convert_image(img_node, translator, src_fname=None):
+ """Convert a image node for the builder.
+
+ Different builder prefer different image formats, e.g. *latex* builder
+ prefer PDF while *html* builder prefer SVG format for images.
+
+ This function handles output image formats in dependence of source the
+ format (of the image) and the translator's output format.
+ """
+ app = translator.builder.app
+
+ fname, in_ext = path.splitext(path.basename(img_node['uri']))
+ if src_fname is None:
+ src_fname = path.join(translator.builder.srcdir, img_node['uri'])
+ if not path.exists(src_fname):
+ src_fname = path.join(translator.builder.outdir, img_node['uri'])
+
+ dst_fname = None
+
+ # in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages
+
+ app.verbose('assert best format for: ' + img_node['uri'])
+
+ if in_ext == '.dot':
+
+ if not dot_cmd:
+ app.verbose("dot from graphviz not available / include DOT raw.")
+ img_node.replace_self(file2literal(src_fname))
+
+ elif translator.builder.format == 'latex':
+ dst_fname = path.join(translator.builder.outdir, fname + '.pdf')
+ img_node['uri'] = fname + '.pdf'
+ img_node['candidates'] = {'*': fname + '.pdf'}
+
+
+ elif translator.builder.format == 'html':
+ dst_fname = path.join(
+ translator.builder.outdir,
+ translator.builder.imagedir,
+ fname + '.svg')
+ img_node['uri'] = path.join(
+ translator.builder.imgpath, fname + '.svg')
+ img_node['candidates'] = {
+ '*': path.join(translator.builder.imgpath, fname + '.svg')}
+
+ else:
+ # all other builder formats will include DOT as raw
+ img_node.replace_self(file2literal(src_fname))
+
+ elif in_ext == '.svg':
+
+ if translator.builder.format == 'latex':
+ if convert_cmd is None:
+ app.verbose("no SVG to PDF conversion available / include SVG raw.")
+ img_node.replace_self(file2literal(src_fname))
+ else:
+ dst_fname = path.join(translator.builder.outdir, fname + '.pdf')
+ img_node['uri'] = fname + '.pdf'
+ img_node['candidates'] = {'*': fname + '.pdf'}
+
+ if dst_fname:
+ # the builder needs not to copy one more time, so pop it if exists.
+ translator.builder.images.pop(img_node['uri'], None)
+ _name = dst_fname[len(translator.builder.outdir) + 1:]
+
+ if isNewer(dst_fname, src_fname):
+ app.verbose("convert: {out}/%s already exists and is newer" % _name)
+
+ else:
+ ok = False
+ mkdir(path.dirname(dst_fname))
+
+ if in_ext == '.dot':
+ app.verbose('convert DOT to: {out}/' + _name)
+ ok = dot2format(app, src_fname, dst_fname)
+
+ elif in_ext == '.svg':
+ app.verbose('convert SVG to: {out}/' + _name)
+ ok = svg2pdf(app, src_fname, dst_fname)
+
+ if not ok:
+ img_node.replace_self(file2literal(src_fname))
+
+
+def dot2format(app, dot_fname, out_fname):
+ """Converts DOT file to ``out_fname`` using ``dot(1)``.
+
+ * ``dot_fname`` pathname of the input DOT file, including extension ``.dot``
+ * ``out_fname`` pathname of the output file, including format extension
+
+ The *format extension* depends on the ``dot`` command (see ``man dot``
+ option ``-Txxx``). Normally you will use one of the following extensions:
+
+ - ``.ps`` for PostScript,
+ - ``.svg`` or ``svgz`` for Structured Vector Graphics,
+ - ``.fig`` for XFIG graphics and
+ - ``.png`` or ``gif`` for common bitmap graphics.
+
+ """
+ out_format = path.splitext(out_fname)[1][1:]
+ cmd = [dot_cmd, '-T%s' % out_format, dot_fname]
+ exit_code = 42
+
+ with open(out_fname, "w") as out:
+ exit_code = subprocess.call(cmd, stdout = out)
+ if exit_code != 0:
+ app.warn("Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
+ return bool(exit_code == 0)
+
+def svg2pdf(app, svg_fname, pdf_fname):
+ """Converts SVG to PDF with ``convert(1)`` command.
+
+ Uses ``convert(1)`` from ImageMagick (https://www.imagemagick.org) for
+ conversion. Returns ``True`` on success and ``False`` if an error occurred.
+
+ * ``svg_fname`` pathname of the input SVG file with extension (``.svg``)
+ * ``pdf_name`` pathname of the output PDF file with extension (``.pdf``)
+
+ """
+ cmd = [convert_cmd, svg_fname, pdf_fname]
+ # use stdout and stderr from parent
+ exit_code = subprocess.call(cmd)
+ if exit_code != 0:
+ app.warn("Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
+ return bool(exit_code == 0)
+
+
+# image handling
+# ---------------------
+
+def visit_kernel_image(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_image`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0]
+ convert_image(img_node, self)
+
+class kernel_image(nodes.image):
+ """Node for ``kernel-image`` directive."""
+ pass
+
+class KernelImage(images.Image):
+ u"""KernelImage directive
+
+ Earns everything from ``.. image::`` directive, except *remote URI* and
+ *glob* pattern. The KernelImage wraps a image node into a
+ kernel_image node. See ``visit_kernel_image``.
+ """
+
+ def run(self):
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s": glob pattern and remote images are not allowed'
+ % (self.name, uri))
+ result = images.Image.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (image_node,) = result
+ # wrap image node into a kernel_image node / see visitors
+ node = kernel_image('', image_node)
+ return [node]
+
+# figure handling
+# ---------------------
+
+def visit_kernel_figure(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_figure`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0][0]
+ convert_image(img_node, self)
+
+class kernel_figure(nodes.figure):
+ """Node for ``kernel-figure`` directive."""
+
+class KernelFigure(Figure):
+ u"""KernelImage directive
+
+ Earns everything from ``.. figure::`` directive, except *remote URI* and
+ *glob* pattern. The KernelFigure wraps a figure node into a kernel_figure
+ node. See ``visit_kernel_figure``.
+ """
+
+ def run(self):
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s":'
+ ' glob pattern and remote images are not allowed'
+ % (self.name, uri))
+ result = Figure.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (figure_node,) = result
+ # wrap figure node into a kernel_figure node / see visitors
+ node = kernel_figure('', figure_node)
+ return [node]
+
+
+# render handling
+# ---------------------
+
+def visit_kernel_render(self, node):
+ """Visitor of the ``kernel_render`` Node.
+
+ If rendering tools available, save the markup of the ``literal_block`` child
+ node into a file and replace the ``literal_block`` node with a new created
+ ``image`` node, pointing to the saved markup file. Afterwards, handle the
+ image child-node with the ``convert_image(...)``.
+ """
+ app = self.builder.app
+ srclang = node.get('srclang')
+
+ app.verbose('visit kernel-render node lang: "%s"' % (srclang))
+
+ tmp_ext = RENDER_MARKUP_EXT.get(srclang, None)
+ if tmp_ext is None:
+ app.warn('kernel-render: "%s" unknow / include raw.' % (srclang))
+ return
+
+ if not dot_cmd and tmp_ext == '.dot':
+ app.verbose("dot from graphviz not available / include raw.")
+ return
+
+ literal_block = node[0]
+
+ code = literal_block.astext()
+ hashobj = code.encode('utf-8') # str(node.attributes)
+ fname = path.join('%s-%s' % (srclang, sha1(hashobj).hexdigest()))
+
+ tmp_fname = path.join(
+ self.builder.outdir, self.builder.imagedir, fname + tmp_ext)
+
+ if not path.isfile(tmp_fname):
+ mkdir(path.dirname(tmp_fname))
+ with open(tmp_fname, "w") as out:
+ out.write(code)
+
+ img_node = nodes.image(node.rawsource, **node.attributes)
+ img_node['uri'] = path.join(self.builder.imgpath, fname + tmp_ext)
+ img_node['candidates'] = {
+ '*': path.join(self.builder.imgpath, fname + tmp_ext)}
+
+ literal_block.replace_self(img_node)
+ convert_image(img_node, self, tmp_fname)
+
+
+class kernel_render(nodes.General, nodes.Inline, nodes.Element):
+ """Node for ``kernel-render`` directive."""
+ pass
+
+class KernelRender(Figure):
+ u"""KernelRender directive
+
+ Render content by external tool. Has all the options known from the
+ *figure* directive, plus option ``caption``. If ``caption`` has a
+ value, a figure node with the *caption* is inserted. If not, a image node is
+ inserted.
+
+ The KernelRender directive wraps the text of the directive into a
+ literal_block node and wraps it into a kernel_render node. See
+ ``visit_kernel_render``.
+ """
+ has_content = True
+ required_arguments = 1
+ optional_arguments = 0
+ final_argument_whitespace = False
+
+ # earn options from 'figure'
+ option_spec = Figure.option_spec.copy()
+ option_spec['caption'] = directives.unchanged
+
+ def run(self):
+ return [self.build_node()]
+
+ def build_node(self):
+
+ srclang = self.arguments[0].strip()
+ if srclang not in RENDER_MARKUP_EXT.keys():
+ return [self.state_machine.reporter.warning(
+ 'Unknow source language "%s", use one of: %s.' % (
+ srclang, ",".join(RENDER_MARKUP_EXT.keys())),
+ line=self.lineno)]
+
+ code = '\n'.join(self.content)
+ if not code.strip():
+ return [self.state_machine.reporter.warning(
+ 'Ignoring "%s" directive without content.' % (
+ self.name),
+ line=self.lineno)]
+
+ node = kernel_render()
+ node['alt'] = self.options.get('alt','')
+ node['srclang'] = srclang
+ literal_node = nodes.literal_block(code, code)
+ node += literal_node
+
+ caption = self.options.get('caption')
+ if caption:
+ # parse caption's content
+ parsed = nodes.Element()
+ self.state.nested_parse(
+ ViewList([caption], source=''), self.content_offset, parsed)
+ caption_node = nodes.caption(
+ parsed[0].rawsource, '', *parsed[0].children)
+ caption_node.source = parsed[0].source
+ caption_node.line = parsed[0].line
+
+ figure_node = nodes.figure('', node)
+ for k,v in self.options.items():
+ figure_node[k] = v
+ figure_node += caption_node
+
+ node = figure_node
+
+ return node
+
+def add_kernel_figure_to_std_domain(app, doctree):
+ """Add kernel-figure anchors to 'std' domain.
+
+ The ``StandardDomain.process_doc(..)`` method does not know how to resolve
+ the caption (label) of ``kernel-figure`` directive (it only knows about
+ standard nodes, e.g. table, figure etc.). Without any additional handling
+ this will result in a 'undefined label' for kernel-figures.
+
+ This handle adds labels of kernel-figure to the 'std' domain labels.
+ """
+
+ std = app.env.domains["std"]
+ docname = app.env.docname
+ labels = std.data["labels"]
+
+ for name, explicit in iteritems(doctree.nametypes):
+ if not explicit:
+ continue
+ labelid = doctree.nameids[name]
+ if labelid is None:
+ continue
+ node = doctree.ids[labelid]
+
+ if node.tagname == 'kernel_figure':
+ for n in node.next_node():
+ if n.tagname == 'caption':
+ sectname = clean_astext(n)
+ # add label to std domain
+ labels[name] = docname, labelid, sectname
+ break
DRM DRIVER FOR BOCHS VIRTUAL GPU
M: Gerd Hoffmann <kraxel@redhat.com>
L: virtualization@lists.linux-foundation.org
-T: git git://git.kraxel.org/linux drm-qemu
+T: git git://anongit.freedesktop.org/drm/drm-misc
S: Maintained
F: drivers/gpu/drm/bochs/
M: Dave Airlie <airlied@redhat.com>
M: Gerd Hoffmann <kraxel@redhat.com>
L: virtualization@lists.linux-foundation.org
-T: git git://git.kraxel.org/linux drm-qemu
+T: git git://anongit.freedesktop.org/drm/drm-misc
S: Obsolete
W: https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/
F: drivers/gpu/drm/cirrus/
S: Supported
F: drivers/gpu/drm/atmel-hlcdc/
F: Documentation/devicetree/bindings/drm/atmel/
+T: git git://anongit.freedesktop.org/drm/drm-misc
DRM DRIVERS FOR ALLWINNER A10
M: Maxime Ripard <maxime.ripard@free-electrons.com>
S: Supported
F: drivers/gpu/drm/meson/
F: Documentation/devicetree/bindings/display/amlogic,meson-vpu.txt
+T: git git://anongit.freedesktop.org/drm/drm-meson
+T: git git://anongit.freedesktop.org/drm/drm-misc
DRM DRIVERS FOR EXYNOS
M: Inki Dae <inki.dae@samsung.com>
M: Dave Airlie <airlied@redhat.com>
M: Gerd Hoffmann <kraxel@redhat.com>
L: virtualization@lists.linux-foundation.org
-T: git git://git.kraxel.org/linux drm-qemu
+T: git git://anongit.freedesktop.org/drm/drm-misc
S: Maintained
F: drivers/gpu/drm/qxl/
F: include/uapi/drm/qxl_drm.h
S: Maintained
F: drivers/gpu/drm/rockchip/
F: Documentation/devicetree/bindings/display/rockchip/
+T: git git://anongit.freedesktop.org/drm/drm-misc
DRM DRIVER FOR SAVAGE VIDEO CARDS
S: Orphan / Obsolete
F: drivers/gpu/drm/vc4/
F: include/uapi/drm/vc4_drm.h
F: Documentation/devicetree/bindings/display/brcm,bcm-vc4.txt
+T: git git://anongit.freedesktop.org/drm/drm-misc
DRM DRIVERS FOR TI OMAP
M: Tomi Valkeinen <tomi.valkeinen@ti.com>
S: Maintained
F: drivers/gpu/drm/zte/
F: Documentation/devicetree/bindings/display/zte,vou.txt
+T: git git://anongit.freedesktop.org/drm/drm-misc
DSBR100 USB FM RADIO DRIVER
M: Alexey Klimov <klimov.linux@gmail.com>
S: Maintained
F: drivers/net/wireless/mediatek/mt7601u/
+MEGACHIPS STDPXXXX-GE-B850V3-FW LVDS/DP++ BRIDGES
+M: Peter Senna Tschudin <peter.senna@collabora.com>
+M: Martin Donnelly <martin.donnelly@ge.com>
+M: Martyn Welch <martyn.welch@collabora.co.uk>
+S: Maintained
+F: drivers/gpu/drm/bridge/megachips-stdpxxxx-ge-b850v3-fw.c
+F: Documentation/devicetree/bindings/video/bridge/megachips-stdpxxxx-ge-b850v3-fw.txt
+
MEGARAID SCSI/SAS DRIVERS
M: Kashyap Desai <kashyap.desai@broadcom.com>
M: Sumit Saxena <sumit.saxena@broadcom.com>
M: Gerd Hoffmann <kraxel@redhat.com>
L: dri-devel@lists.freedesktop.org
L: virtualization@lists.linux-foundation.org
-T: git git://git.kraxel.org/linux drm-qemu
+T: git git://anongit.freedesktop.org/drm/drm-misc
S: Maintained
F: drivers/gpu/drm/virtio/
F: include/uapi/linux/virtio_gpu.h
#ifndef IOSF_MBI_SYMS_H
#define IOSF_MBI_SYMS_H
+#include <linux/notifier.h>
+
#define MBI_MCR_OFFSET 0xD0
#define MBI_MDR_OFFSET 0xD4
#define MBI_MCRX_OFFSET 0xD8
#define QRK_MBI_UNIT_MM 0x05
#define QRK_MBI_UNIT_SOC 0x31
+/* Action values for the pmic_bus_access_notifier functions */
+#define MBI_PMIC_BUS_ACCESS_BEGIN 1
+#define MBI_PMIC_BUS_ACCESS_END 2
+
#if IS_ENABLED(CONFIG_IOSF_MBI)
bool iosf_mbi_available(void);
*/
int iosf_mbi_modify(u8 port, u8 opcode, u32 offset, u32 mdr, u32 mask);
+/**
+ * iosf_mbi_punit_acquire() - Acquire access to the P-Unit
+ *
+ * One some systems the P-Unit accesses the PMIC to change various voltages
+ * through the same bus as other kernel drivers use for e.g. battery monitoring.
+ *
+ * If a driver sends requests to the P-Unit which require the P-Unit to access
+ * the PMIC bus while another driver is also accessing the PMIC bus various bad
+ * things happen.
+ *
+ * To avoid these problems this function must be called before accessing the
+ * P-Unit or the PMIC, be it through iosf_mbi* functions or through other means.
+ *
+ * Note on these systems the i2c-bus driver will request a sempahore from the
+ * P-Unit for exclusive access to the PMIC bus when i2c drivers are accessing
+ * it, but this does not appear to be sufficient, we still need to avoid making
+ * certain P-Unit requests during the access window to avoid problems.
+ *
+ * This function locks a mutex, as such it may sleep.
+ */
+void iosf_mbi_punit_acquire(void);
+
+/**
+ * iosf_mbi_punit_release() - Release access to the P-Unit
+ */
+void iosf_mbi_punit_release(void);
+
+/**
+ * iosf_mbi_register_pmic_bus_access_notifier - Register PMIC bus notifier
+ *
+ * This function can be used by drivers which may need to acquire P-Unit
+ * managed resources from interrupt context, where iosf_mbi_punit_acquire()
+ * can not be used.
+ *
+ * This function allows a driver to register a notifier to get notified (in a
+ * process context) before other drivers start accessing the PMIC bus.
+ *
+ * This allows the driver to acquire any resources, which it may need during
+ * the window the other driver is accessing the PMIC, before hand.
+ *
+ * @nb: notifier_block to register
+ */
+int iosf_mbi_register_pmic_bus_access_notifier(struct notifier_block *nb);
+
+/**
+ * iosf_mbi_register_pmic_bus_access_notifier - Unregister PMIC bus notifier
+ *
+ * @nb: notifier_block to unregister
+ */
+int iosf_mbi_unregister_pmic_bus_access_notifier(struct notifier_block *nb);
+
+/**
+ * iosf_mbi_call_pmic_bus_access_notifier_chain - Call PMIC bus notifier chain
+ *
+ * @val: action to pass into listener's notifier_call function
+ * @v: data pointer to pass into listener's notifier_call function
+ */
+int iosf_mbi_call_pmic_bus_access_notifier_chain(unsigned long val, void *v);
+
#else /* CONFIG_IOSF_MBI is not enabled */
static inline
bool iosf_mbi_available(void)
WARN(1, "IOSF_MBI driver not available");
return -EPERM;
}
+
+static inline void iosf_mbi_punit_acquire(void) {}
+static inline void iosf_mbi_punit_release(void) {}
+
+static inline
+int iosf_mbi_register_pmic_bus_access_notifier(struct notifier_block *nb)
+{
+ return 0;
+}
+
+static inline
+int iosf_mbi_unregister_pmic_bus_access_notifier(struct notifier_block *nb)
+{
+ return 0;
+}
+
+static inline
+int iosf_mbi_call_pmic_bus_access_notifier_chain(unsigned long val, void *v)
+{
+ return 0;
+}
+
#endif /* CONFIG_IOSF_MBI */
#endif /* IOSF_MBI_SYMS_H */
INTEL_SKL_IDS(&gen9_early_ops),
INTEL_BXT_IDS(&gen9_early_ops),
INTEL_KBL_IDS(&gen9_early_ops),
+ INTEL_GLK_IDS(&gen9_early_ops),
};
static void __init
static struct pci_dev *mbi_pdev;
static DEFINE_SPINLOCK(iosf_mbi_lock);
+static DEFINE_MUTEX(iosf_mbi_punit_mutex);
+static BLOCKING_NOTIFIER_HEAD(iosf_mbi_pmic_bus_access_notifier);
static inline u32 iosf_mbi_form_mcr(u8 op, u8 port, u8 offset)
{
}
EXPORT_SYMBOL(iosf_mbi_available);
+void iosf_mbi_punit_acquire(void)
+{
+ mutex_lock(&iosf_mbi_punit_mutex);
+}
+EXPORT_SYMBOL(iosf_mbi_punit_acquire);
+
+void iosf_mbi_punit_release(void)
+{
+ mutex_unlock(&iosf_mbi_punit_mutex);
+}
+EXPORT_SYMBOL(iosf_mbi_punit_release);
+
+int iosf_mbi_register_pmic_bus_access_notifier(struct notifier_block *nb)
+{
+ int ret;
+
+ /* Wait for the bus to go inactive before registering */
+ mutex_lock(&iosf_mbi_punit_mutex);
+ ret = blocking_notifier_chain_register(
+ &iosf_mbi_pmic_bus_access_notifier, nb);
+ mutex_unlock(&iosf_mbi_punit_mutex);
+
+ return ret;
+}
+EXPORT_SYMBOL(iosf_mbi_register_pmic_bus_access_notifier);
+
+int iosf_mbi_unregister_pmic_bus_access_notifier(struct notifier_block *nb)
+{
+ int ret;
+
+ /* Wait for the bus to go inactive before unregistering */
+ mutex_lock(&iosf_mbi_punit_mutex);
+ ret = blocking_notifier_chain_unregister(
+ &iosf_mbi_pmic_bus_access_notifier, nb);
+ mutex_unlock(&iosf_mbi_punit_mutex);
+
+ return ret;
+}
+EXPORT_SYMBOL(iosf_mbi_unregister_pmic_bus_access_notifier);
+
+int iosf_mbi_call_pmic_bus_access_notifier_chain(unsigned long val, void *v)
+{
+ return blocking_notifier_call_chain(
+ &iosf_mbi_pmic_bus_access_notifier, val, v);
+}
+EXPORT_SYMBOL(iosf_mbi_call_pmic_bus_access_notifier_chain);
+
#ifdef CONFIG_IOSF_MBI_DEBUG
static u32 dbg_mdr;
static u32 dbg_mcr;
writel_relaxed(addr | pte_flags, intel_private.gtt + entry);
}
-static const struct aper_size_info_fixed intel_fake_agp_sizes[] = {
- {32, 8192, 3},
- {64, 16384, 4},
- {128, 32768, 5},
- {256, 65536, 6},
- {512, 131072, 7},
-};
-
static unsigned int intel_gtt_stolen_size(void)
{
u16 gmch_ctrl;
}
#if IS_ENABLED(CONFIG_AGP_INTEL)
+static const struct aper_size_info_fixed intel_fake_agp_sizes[] = {
+ {32, 8192, 3},
+ {64, 16384, 4},
+ {128, 32768, 5},
+ {256, 65536, 6},
+ {512, 131072, 7},
+};
+
static int intel_fake_agp_fetch_size(void)
{
int num_sizes = ARRAY_SIZE(intel_fake_agp_sizes);
return array;
}
EXPORT_SYMBOL(dma_fence_array_create);
+
+/**
+ * dma_fence_match_context - Check if all fences are from the given context
+ * @fence: [in] fence or fence array
+ * @context: [in] fence context to check all fences against
+ *
+ * Checks the provided fence or, for a fence array, all fences in the array
+ * against the given context. Returns false if any fence is from a different
+ * context.
+ */
+bool dma_fence_match_context(struct dma_fence *fence, u64 context)
+{
+ struct dma_fence_array *array = to_dma_fence_array(fence);
+ unsigned i;
+
+ if (!dma_fence_is_array(fence))
+ return fence->context == context;
+
+ for (i = 0; i < array->num_fences; i++) {
+ if (array->fences[i]->context != context)
+ return false;
+ }
+
+ return true;
+}
+EXPORT_SYMBOL(dma_fence_match_context);
* after it signals with dma_fence_signal. The callback itself can be called
* from irq context.
*
+ * Returns 0 in case of success, -ENOENT if the fence is already signaled
+ * and -EINVAL in case of error.
*/
int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb,
dma_fence_func_t func)
If in doubt, say "Y".
+config DRM_FBDEV_OVERALLOC
+ int "Overallocation of the fbdev buffer"
+ depends on DRM_FBDEV_EMULATION
+ default 100
+ help
+ Defines the fbdev buffer overallocation in percent. Default
+ is 100. Typical values for double buffering will be 200,
+ triple buffering 300.
+
config DRM_LOAD_EDID_FIRMWARE
bool "Allow to specify an EDID data set instead of probing for it"
depends on DRM_KMS_HELPER
drm-y := drm_auth.o drm_bufs.o drm_cache.o \
drm_context.o drm_dma.o \
- drm_fops.o drm_gem.o drm_ioctl.o drm_irq.o \
+ drm_file.o drm_gem.o drm_ioctl.o drm_irq.o \
drm_lock.o drm_memory.o drm_drv.o \
drm_scatter.o drm_pci.o \
- drm_platform.o drm_sysfs.o drm_hashtab.o drm_mm.o \
+ drm_sysfs.o drm_hashtab.o drm_mm.o \
drm_crtc.o drm_fourcc.o drm_modes.o drm_edid.o \
drm_info.o drm_encoder_slave.o \
drm_trace_points.o drm_global.o drm_prime.o \
drm_kms_helper-y := drm_crtc_helper.o drm_dp_helper.o drm_probe_helper.o \
drm_plane_helper.o drm_dp_mst_topology.o drm_atomic_helper.o \
drm_kms_helper_common.o drm_dp_dual_mode_helper.o \
- drm_simple_kms_helper.o drm_modeset_helper.o
+ drm_simple_kms_helper.o drm_modeset_helper.o \
+ drm_scdc_helper.o
drm_kms_helper-$(CONFIG_DRM_LOAD_EDID_FIRMWARE) += drm_edid_load.o
drm_kms_helper-$(CONFIG_DRM_FBDEV_EMULATION) += drm_fb_helper.o
info = drm_fb_helper_alloc_fbi(helper);
if (IS_ERR(info)) {
ret = PTR_ERR(info);
- goto out_unref;
+ goto out;
}
info->par = rfbdev;
ret = amdgpu_framebuffer_init(adev->ddev, &rfbdev->rfb, &mode_cmd, gobj);
if (ret) {
DRM_ERROR("failed to initialize framebuffer %d\n", ret);
- goto out_destroy_fbi;
+ goto out;
}
fb = &rfbdev->rfb.base;
if (info->screen_base == NULL) {
ret = -ENOSPC;
- goto out_destroy_fbi;
+ goto out;
}
DRM_INFO("fb mappable at 0x%lX\n", info->fix.smem_start);
vga_switcheroo_client_fb_set(adev->ddev->pdev, info);
return 0;
-out_destroy_fbi:
- drm_fb_helper_release_fbi(helper);
-out_unref:
+out:
if (abo) {
}
struct amdgpu_framebuffer *rfb = &rfbdev->rfb;
drm_fb_helper_unregister_fbi(&rfbdev->helper);
- drm_fb_helper_release_fbi(&rfbdev->helper);
if (rfb->obj) {
amdgpufb_destroy_pinned_object(rfb->obj);
.dumb_create = drm_gem_cma_dumb_create,
.dumb_map_offset = drm_gem_cma_dumb_map_offset,
.dumb_destroy = drm_gem_dumb_destroy,
- .get_vblank_counter = drm_vblank_no_hw_counter,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
.gem_free_object_unlocked = drm_gem_cma_free_object,
drm_crtc_cleanup(crtc);
}
+static int hdlcd_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ struct hdlcd_drm_private *hdlcd = crtc_to_hdlcd_priv(crtc);
+ unsigned int mask = hdlcd_read(hdlcd, HDLCD_REG_INT_MASK);
+
+ hdlcd_write(hdlcd, HDLCD_REG_INT_MASK, mask | HDLCD_INTERRUPT_VSYNC);
+
+ return 0;
+}
+
+static void hdlcd_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+ struct hdlcd_drm_private *hdlcd = crtc_to_hdlcd_priv(crtc);
+ unsigned int mask = hdlcd_read(hdlcd, HDLCD_REG_INT_MASK);
+
+ hdlcd_write(hdlcd, HDLCD_REG_INT_MASK, mask & ~HDLCD_INTERRUPT_VSYNC);
+}
+
static const struct drm_crtc_funcs hdlcd_crtc_funcs = {
.destroy = hdlcd_crtc_cleanup,
.set_config = drm_atomic_helper_set_config,
.reset = drm_atomic_helper_crtc_reset,
.atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .enable_vblank = hdlcd_crtc_enable_vblank,
+ .disable_vblank = hdlcd_crtc_disable_vblank,
};
static struct simplefb_format supported_formats[] = SIMPLEFB_FORMATS;
hdlcd_write(hdlcd, HDLCD_REG_INT_MASK, irq_mask);
}
-static int hdlcd_enable_vblank(struct drm_device *drm, unsigned int crtc)
-{
- struct hdlcd_drm_private *hdlcd = drm->dev_private;
- unsigned int mask = hdlcd_read(hdlcd, HDLCD_REG_INT_MASK);
-
- hdlcd_write(hdlcd, HDLCD_REG_INT_MASK, mask | HDLCD_INTERRUPT_VSYNC);
-
- return 0;
-}
-
-static void hdlcd_disable_vblank(struct drm_device *drm, unsigned int crtc)
-{
- struct hdlcd_drm_private *hdlcd = drm->dev_private;
- unsigned int mask = hdlcd_read(hdlcd, HDLCD_REG_INT_MASK);
-
- hdlcd_write(hdlcd, HDLCD_REG_INT_MASK, mask & ~HDLCD_INTERRUPT_VSYNC);
-}
-
#ifdef CONFIG_DEBUG_FS
static int hdlcd_show_underrun_count(struct seq_file *m, void *arg)
{
}
#endif
-static const struct file_operations fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .poll = drm_poll,
- .read = drm_read,
- .llseek = noop_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(fops);
static struct drm_driver hdlcd_driver = {
.driver_features = DRIVER_HAVE_IRQ | DRIVER_GEM |
.irq_preinstall = hdlcd_irq_preinstall,
.irq_postinstall = hdlcd_irq_postinstall,
.irq_uninstall = hdlcd_irq_uninstall,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = hdlcd_enable_vblank,
- .disable_vblank = hdlcd_disable_vblank,
.gem_free_object_unlocked = drm_gem_cma_free_object,
.gem_vm_ops = &drm_gem_cma_vm_ops,
.dumb_create = drm_gem_cma_dumb_create,
.atomic_check = malidp_crtc_atomic_check,
};
+static int malidp_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ struct malidp_drm *malidp = crtc_to_malidp_device(crtc);
+ struct malidp_hw_device *hwdev = malidp->dev;
+
+ malidp_hw_enable_irq(hwdev, MALIDP_DE_BLOCK,
+ hwdev->map.de_irq_map.vsync_irq);
+ return 0;
+}
+
+static void malidp_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+ struct malidp_drm *malidp = crtc_to_malidp_device(crtc);
+ struct malidp_hw_device *hwdev = malidp->dev;
+
+ malidp_hw_disable_irq(hwdev, MALIDP_DE_BLOCK,
+ hwdev->map.de_irq_map.vsync_irq);
+}
+
static const struct drm_crtc_funcs malidp_crtc_funcs = {
.destroy = drm_crtc_cleanup,
.set_config = drm_atomic_helper_set_config,
.reset = drm_atomic_helper_crtc_reset,
.atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .enable_vblank = malidp_crtc_enable_vblank,
+ .disable_vblank = malidp_crtc_disable_vblank,
};
int malidp_crtc_init(struct drm_device *drm)
drm_atomic_helper_cleanup_planes(drm, state);
}
-static struct drm_mode_config_helper_funcs malidp_mode_config_helpers = {
+static const struct drm_mode_config_helper_funcs malidp_mode_config_helpers = {
.atomic_commit_tail = malidp_atomic_commit_tail,
};
.atomic_commit = drm_atomic_helper_commit,
};
-static int malidp_enable_vblank(struct drm_device *drm, unsigned int crtc)
-{
- struct malidp_drm *malidp = drm->dev_private;
- struct malidp_hw_device *hwdev = malidp->dev;
-
- malidp_hw_enable_irq(hwdev, MALIDP_DE_BLOCK,
- hwdev->map.de_irq_map.vsync_irq);
- return 0;
-}
-
-static void malidp_disable_vblank(struct drm_device *drm, unsigned int pipe)
-{
- struct malidp_drm *malidp = drm->dev_private;
- struct malidp_hw_device *hwdev = malidp->dev;
-
- malidp_hw_disable_irq(hwdev, MALIDP_DE_BLOCK,
- hwdev->map.de_irq_map.vsync_irq);
-}
-
static int malidp_init(struct drm_device *drm)
{
int ret;
drm_fbdev_cma_restore_mode(malidp->fbdev);
}
-static const struct file_operations fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .poll = drm_poll,
- .read = drm_read,
- .llseek = noop_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(fops);
static struct drm_driver malidp_driver = {
.driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_ATOMIC |
DRIVER_PRIME,
.lastclose = malidp_lastclose,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = malidp_enable_vblank,
- .disable_vblank = malidp_disable_vblank,
.gem_free_object_unlocked = drm_gem_cma_free_object,
.gem_vm_ops = &drm_gem_cma_vm_ops,
.dumb_create = drm_gem_cma_dumb_create,
return true;
}
+/* These are locked by dev->vbl_lock */
+static void armada_drm_crtc_disable_irq(struct armada_crtc *dcrtc, u32 mask)
+{
+ if (dcrtc->irq_ena & mask) {
+ dcrtc->irq_ena &= ~mask;
+ writel(dcrtc->irq_ena, dcrtc->base + LCD_SPU_IRQ_ENA);
+ }
+}
+
+static void armada_drm_crtc_enable_irq(struct armada_crtc *dcrtc, u32 mask)
+{
+ if ((dcrtc->irq_ena & mask) != mask) {
+ dcrtc->irq_ena |= mask;
+ writel(dcrtc->irq_ena, dcrtc->base + LCD_SPU_IRQ_ENA);
+ if (readl_relaxed(dcrtc->base + LCD_SPU_IRQ_ISR) & mask)
+ writel(0, dcrtc->base + LCD_SPU_IRQ_ISR);
+ }
+}
+
static void armada_drm_crtc_irq(struct armada_crtc *dcrtc, u32 stat)
{
void __iomem *base = dcrtc->base;
return IRQ_NONE;
}
-/* These are locked by dev->vbl_lock */
-void armada_drm_crtc_disable_irq(struct armada_crtc *dcrtc, u32 mask)
-{
- if (dcrtc->irq_ena & mask) {
- dcrtc->irq_ena &= ~mask;
- writel(dcrtc->irq_ena, dcrtc->base + LCD_SPU_IRQ_ENA);
- }
-}
-
-void armada_drm_crtc_enable_irq(struct armada_crtc *dcrtc, u32 mask)
-{
- if ((dcrtc->irq_ena & mask) != mask) {
- dcrtc->irq_ena |= mask;
- writel(dcrtc->irq_ena, dcrtc->base + LCD_SPU_IRQ_ENA);
- if (readl_relaxed(dcrtc->base + LCD_SPU_IRQ_ISR) & mask)
- writel(0, dcrtc->base + LCD_SPU_IRQ_ISR);
- }
-}
-
static uint32_t armada_drm_crtc_calculate_csc(struct armada_crtc *dcrtc)
{
struct drm_display_mode *adj = &dcrtc->crtc.mode;
return 0;
}
+/* These are called under the vbl_lock. */
+static int armada_drm_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ struct armada_crtc *dcrtc = drm_to_armada_crtc(crtc);
+
+ armada_drm_crtc_enable_irq(dcrtc, VSYNC_IRQ_ENA);
+ return 0;
+}
+
+static void armada_drm_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+ struct armada_crtc *dcrtc = drm_to_armada_crtc(crtc);
+
+ armada_drm_crtc_disable_irq(dcrtc, VSYNC_IRQ_ENA);
+}
+
static const struct drm_crtc_funcs armada_crtc_funcs = {
.cursor_set = armada_drm_crtc_cursor_set,
.cursor_move = armada_drm_crtc_cursor_move,
.set_config = drm_crtc_helper_set_config,
.page_flip = armada_drm_crtc_page_flip,
.set_property = armada_drm_crtc_set_property,
+ .enable_vblank = armada_drm_crtc_enable_vblank,
+ .disable_vblank = armada_drm_crtc_disable_vblank,
};
static const struct drm_plane_funcs armada_primary_plane_funcs = {
void armada_drm_crtc_gamma_set(struct drm_crtc *, u16, u16, u16, int);
void armada_drm_crtc_gamma_get(struct drm_crtc *, u16 *, u16 *, u16 *, int);
-void armada_drm_crtc_disable_irq(struct armada_crtc *, u32);
-void armada_drm_crtc_enable_irq(struct armada_crtc *, u32);
void armada_drm_crtc_update_regs(struct armada_crtc *, struct armada_regs *);
void armada_drm_crtc_plane_disable(struct armada_crtc *dcrtc,
};
#define ARMADA_DEBUGFS_ENTRIES ARRAY_SIZE(armada_debugfs_list)
-static int drm_add_fake_info_node(struct drm_minor *minor, struct dentry *ent,
- const void *key)
-{
- struct drm_info_node *node;
-
- node = kmalloc(sizeof(struct drm_info_node), GFP_KERNEL);
- if (!node) {
- debugfs_remove(ent);
- return -ENOMEM;
- }
-
- node->minor = minor;
- node->dent = ent;
- node->info_ent = (void *) key;
-
- mutex_lock(&minor->debugfs_lock);
- list_add(&node->list, &minor->debugfs_list);
- mutex_unlock(&minor->debugfs_lock);
-
- return 0;
-}
-
-static int armada_debugfs_create(struct dentry *root, struct drm_minor *minor,
- const char *name, umode_t mode, const struct file_operations *fops)
-{
- struct dentry *de;
-
- de = debugfs_create_file(name, mode, root, minor->dev, fops);
-
- return drm_add_fake_info_node(minor, de, fops);
-}
-
int armada_drm_debugfs_init(struct drm_minor *minor)
{
+ struct dentry *de;
int ret;
ret = drm_debugfs_create_files(armada_debugfs_list,
if (ret)
return ret;
- ret = armada_debugfs_create(minor->debugfs_root, minor,
- "reg", S_IFREG | S_IRUSR, &fops_reg_r);
- if (ret)
- goto err_1;
+ de = debugfs_create_file("reg", S_IFREG | S_IRUSR,
+ minor->debugfs_root, minor->dev, &fops_reg_r);
+ if (!de)
+ return -ENOMEM;
- ret = armada_debugfs_create(minor->debugfs_root, minor,
- "reg_wr", S_IFREG | S_IWUSR, &fops_reg_w);
- if (ret)
- goto err_2;
- return ret;
-
- err_2:
- drm_debugfs_remove_files((struct drm_info_list *)&fops_reg_r, 1, minor);
- err_1:
- drm_debugfs_remove_files(armada_debugfs_list, ARMADA_DEBUGFS_ENTRIES,
- minor);
- return ret;
-}
+ de = debugfs_create_file("reg_wr", S_IFREG | S_IWUSR,
+ minor->debugfs_root, minor->dev, &fops_reg_w);
+ if (!de)
+ return -ENOMEM;
-void armada_drm_debugfs_cleanup(struct drm_minor *minor)
-{
- drm_debugfs_remove_files((struct drm_info_list *)&fops_reg_w, 1, minor);
- drm_debugfs_remove_files((struct drm_info_list *)&fops_reg_r, 1, minor);
- drm_debugfs_remove_files(armada_debugfs_list, ARMADA_DEBUGFS_ENTRIES,
- minor);
+ return 0;
}
int armada_overlay_plane_create(struct drm_device *, unsigned long);
int armada_drm_debugfs_init(struct drm_minor *);
-void armada_drm_debugfs_cleanup(struct drm_minor *);
#endif
spin_unlock_irqrestore(&dev->event_lock, flags);
}
-/* These are called under the vbl_lock. */
-static int armada_drm_enable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- struct armada_private *priv = dev->dev_private;
- armada_drm_crtc_enable_irq(priv->dcrtc[pipe], VSYNC_IRQ_ENA);
- return 0;
-}
-
-static void armada_drm_disable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- struct armada_private *priv = dev->dev_private;
- armada_drm_crtc_disable_irq(priv->dcrtc[pipe], VSYNC_IRQ_ENA);
-}
-
static struct drm_ioctl_desc armada_ioctls[] = {
DRM_IOCTL_DEF_DRV(ARMADA_GEM_CREATE, armada_gem_create_ioctl,0),
DRM_IOCTL_DEF_DRV(ARMADA_GEM_MMAP, armada_gem_mmap_ioctl, 0),
armada_fbdev_lastclose(dev);
}
-static const struct file_operations armada_drm_fops = {
- .owner = THIS_MODULE,
- .llseek = no_llseek,
- .read = drm_read,
- .poll = drm_poll,
- .unlocked_ioctl = drm_ioctl,
- .mmap = drm_gem_mmap,
- .open = drm_open,
- .release = drm_release,
-};
+DEFINE_DRM_GEM_FOPS(armada_drm_fops);
static struct drm_driver armada_drm_driver = {
.lastclose = armada_drm_lastclose,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = armada_drm_enable_vblank,
- .disable_vblank = armada_drm_disable_vblank,
.gem_free_object_unlocked = armada_gem_free_object,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
return ret;
}
- priv->drm.platformdev = to_platform_device(dev);
priv->drm.dev_private = priv;
- platform_set_drvdata(priv->drm.platformdev, &priv->drm);
+ dev_set_drvdata(dev, &priv->drm);
INIT_WORK(&priv->fb_unref_work, armada_drm_unref_work);
INIT_KFIFO(priv->fb_unref);
drm_kms_helper_poll_fini(&priv->drm);
armada_fbdev_fini(&priv->drm);
-#ifdef CONFIG_DEBUG_FS
- armada_drm_debugfs_cleanup(priv->drm.primary);
-#endif
drm_dev_unregister(&priv->drm);
component_unbind_all(dev, &priv->drm);
return 0;
err_fb_setup:
- drm_fb_helper_release_fbi(fbh);
drm_fb_helper_fini(fbh);
err_fb_helper:
priv->fbdev = NULL;
if (fbh) {
drm_fb_helper_unregister_fbi(fbh);
- drm_fb_helper_release_fbi(fbh);
drm_fb_helper_fini(fbh);
info = drm_fb_helper_alloc_fbi(helper);
if (IS_ERR(info)) {
ret = PTR_ERR(info);
- goto err_free_vram;
+ goto out;
}
info->par = afbdev;
ret = ast_framebuffer_init(dev, &afbdev->afb, &mode_cmd, gobj);
if (ret)
- goto err_release_fbi;
+ goto out;
afbdev->sysram = sysram;
afbdev->size = size;
return 0;
-err_release_fbi:
- drm_fb_helper_release_fbi(helper);
-err_free_vram:
+out:
vfree(sysram);
return ret;
}
struct ast_framebuffer *afb = &afbdev->afb;
drm_fb_helper_unregister_fbi(&afbdev->helper);
- drm_fb_helper_release_fbi(&afbdev->helper);
if (afb->obj) {
drm_gem_object_unreference_unlocked(afb->obj);
atmel-hlcdc-dc-y := atmel_hlcdc_crtc.o \
atmel_hlcdc_dc.o \
- atmel_hlcdc_layer.o \
atmel_hlcdc_output.o \
atmel_hlcdc_plane.o
* @hlcdc: pointer to the atmel_hlcdc structure provided by the MFD device
* @event: pointer to the current page flip event
* @id: CRTC id (returned by drm_crtc_index)
- * @enabled: CRTC state
*/
struct atmel_hlcdc_crtc {
struct drm_crtc base;
struct atmel_hlcdc_dc *dc;
struct drm_pending_vblank_event *event;
int id;
- bool enabled;
};
static inline struct atmel_hlcdc_crtc *
struct regmap *regmap = crtc->dc->hlcdc->regmap;
unsigned int status;
- if (!crtc->enabled)
- return;
-
drm_crtc_vblank_off(c);
pm_runtime_get_sync(dev->dev);
pm_runtime_allow(dev->dev);
pm_runtime_put_sync(dev->dev);
-
- crtc->enabled = false;
}
static void atmel_hlcdc_crtc_enable(struct drm_crtc *c)
struct regmap *regmap = crtc->dc->hlcdc->regmap;
unsigned int status;
- if (crtc->enabled)
- return;
-
pm_runtime_get_sync(dev->dev);
pm_runtime_forbid(dev->dev);
pm_runtime_put_sync(dev->dev);
drm_crtc_vblank_on(c);
-
- crtc->enabled = true;
-}
-
-void atmel_hlcdc_crtc_suspend(struct drm_crtc *c)
-{
- struct atmel_hlcdc_crtc *crtc = drm_crtc_to_atmel_hlcdc_crtc(c);
-
- if (crtc->enabled) {
- atmel_hlcdc_crtc_disable(c);
- /* save enable state for resume */
- crtc->enabled = true;
- }
-}
-
-void atmel_hlcdc_crtc_resume(struct drm_crtc *c)
-{
- struct atmel_hlcdc_crtc *crtc = drm_crtc_to_atmel_hlcdc_crtc(c);
-
- if (crtc->enabled) {
- crtc->enabled = false;
- atmel_hlcdc_crtc_enable(c);
- }
}
#define ATMEL_HLCDC_RGB444_OUTPUT BIT(0)
kfree(state);
}
+static int atmel_hlcdc_crtc_enable_vblank(struct drm_crtc *c)
+{
+ struct atmel_hlcdc_crtc *crtc = drm_crtc_to_atmel_hlcdc_crtc(c);
+ struct regmap *regmap = crtc->dc->hlcdc->regmap;
+
+ /* Enable SOF (Start Of Frame) interrupt for vblank counting */
+ regmap_write(regmap, ATMEL_HLCDC_IER, ATMEL_HLCDC_SOF);
+
+ return 0;
+}
+
+static void atmel_hlcdc_crtc_disable_vblank(struct drm_crtc *c)
+{
+ struct atmel_hlcdc_crtc *crtc = drm_crtc_to_atmel_hlcdc_crtc(c);
+ struct regmap *regmap = crtc->dc->hlcdc->regmap;
+
+ regmap_write(regmap, ATMEL_HLCDC_IDR, ATMEL_HLCDC_SOF);
+}
+
static const struct drm_crtc_funcs atmel_hlcdc_crtc_funcs = {
.page_flip = drm_atomic_helper_page_flip,
.set_config = drm_atomic_helper_set_config,
.reset = atmel_hlcdc_crtc_reset,
.atomic_duplicate_state = atmel_hlcdc_crtc_duplicate_state,
.atomic_destroy_state = atmel_hlcdc_crtc_destroy_state,
+ .enable_vblank = atmel_hlcdc_crtc_enable_vblank,
+ .disable_vblank = atmel_hlcdc_crtc_disable_vblank,
};
int atmel_hlcdc_crtc_create(struct drm_device *dev)
{
+ struct atmel_hlcdc_plane *primary = NULL, *cursor = NULL;
struct atmel_hlcdc_dc *dc = dev->dev_private;
- struct atmel_hlcdc_planes *planes = dc->planes;
struct atmel_hlcdc_crtc *crtc;
int ret;
int i;
crtc->dc = dc;
- ret = drm_crtc_init_with_planes(dev, &crtc->base,
- &planes->primary->base,
- planes->cursor ? &planes->cursor->base : NULL,
- &atmel_hlcdc_crtc_funcs, NULL);
+ for (i = 0; i < ATMEL_HLCDC_MAX_LAYERS; i++) {
+ if (!dc->layers[i])
+ continue;
+
+ switch (dc->layers[i]->desc->type) {
+ case ATMEL_HLCDC_BASE_LAYER:
+ primary = atmel_hlcdc_layer_to_plane(dc->layers[i]);
+ break;
+
+ case ATMEL_HLCDC_CURSOR_LAYER:
+ cursor = atmel_hlcdc_layer_to_plane(dc->layers[i]);
+ break;
+
+ default:
+ break;
+ }
+ }
+
+ ret = drm_crtc_init_with_planes(dev, &crtc->base, &primary->base,
+ &cursor->base, &atmel_hlcdc_crtc_funcs,
+ NULL);
if (ret < 0)
goto fail;
crtc->id = drm_crtc_index(&crtc->base);
- if (planes->cursor)
- planes->cursor->base.possible_crtcs = 1 << crtc->id;
+ for (i = 0; i < ATMEL_HLCDC_MAX_LAYERS; i++) {
+ struct atmel_hlcdc_plane *overlay;
- for (i = 0; i < planes->noverlays; i++)
- planes->overlays[i]->base.possible_crtcs = 1 << crtc->id;
+ if (dc->layers[i] &&
+ dc->layers[i]->desc->type == ATMEL_HLCDC_OVERLAY_LAYER) {
+ overlay = atmel_hlcdc_layer_to_plane(dc->layers[i]);
+ overlay->base.possible_crtcs = 1 << crtc->id;
+ }
+ }
drm_crtc_helper_add(&crtc->base, &lcdc_crtc_helper_funcs);
drm_crtc_vblank_reset(&crtc->base);
.regs_offset = 0x40,
.id = 0,
.type = ATMEL_HLCDC_BASE_LAYER,
- .nconfigs = 5,
+ .cfgs_offset = 0x2c,
.layout = {
.xstride = { 2 },
.default_color = 3,
.regs_offset = 0x40,
.id = 0,
.type = ATMEL_HLCDC_BASE_LAYER,
- .nconfigs = 5,
+ .cfgs_offset = 0x2c,
.layout = {
.xstride = { 2 },
.default_color = 3,
.regs_offset = 0x100,
.id = 1,
.type = ATMEL_HLCDC_OVERLAY_LAYER,
- .nconfigs = 10,
+ .cfgs_offset = 0x2c,
.layout = {
.pos = 2,
.size = 3,
.regs_offset = 0x280,
.id = 2,
.type = ATMEL_HLCDC_OVERLAY_LAYER,
- .nconfigs = 17,
+ .cfgs_offset = 0x4c,
.layout = {
.pos = 2,
.size = 3,
.chroma_key = 10,
.chroma_key_mask = 11,
.general_config = 12,
+ .scaler_config = 13,
.csc = 14,
},
},
.regs_offset = 0x340,
.id = 3,
.type = ATMEL_HLCDC_CURSOR_LAYER,
- .nconfigs = 10,
.max_width = 128,
.max_height = 128,
+ .cfgs_offset = 0x2c,
.layout = {
.pos = 2,
.size = 3,
.regs_offset = 0x40,
.id = 0,
.type = ATMEL_HLCDC_BASE_LAYER,
- .nconfigs = 7,
+ .cfgs_offset = 0x2c,
.layout = {
.xstride = { 2 },
.default_color = 3,
.regs_offset = 0x140,
.id = 1,
.type = ATMEL_HLCDC_OVERLAY_LAYER,
- .nconfigs = 10,
+ .cfgs_offset = 0x2c,
.layout = {
.pos = 2,
.size = 3,
.regs_offset = 0x240,
.id = 2,
.type = ATMEL_HLCDC_OVERLAY_LAYER,
- .nconfigs = 10,
+ .cfgs_offset = 0x2c,
.layout = {
.pos = 2,
.size = 3,
.regs_offset = 0x340,
.id = 3,
.type = ATMEL_HLCDC_OVERLAY_LAYER,
- .nconfigs = 42,
+ .cfgs_offset = 0x4c,
.layout = {
.pos = 2,
.size = 3,
.chroma_key = 10,
.chroma_key_mask = 11,
.general_config = 12,
+ .scaler_config = 13,
+ .phicoeffs = {
+ .x = 17,
+ .y = 33,
+ },
.csc = 14,
},
},
.regs_offset = 0x440,
.id = 4,
.type = ATMEL_HLCDC_CURSOR_LAYER,
- .nconfigs = 10,
.max_width = 128,
.max_height = 128,
+ .cfgs_offset = 0x2c,
.layout = {
.pos = 2,
.size = 3,
.chroma_key = 7,
.chroma_key_mask = 8,
.general_config = 9,
+ .scaler_config = 13,
},
},
};
.regs_offset = 0x40,
.id = 0,
.type = ATMEL_HLCDC_BASE_LAYER,
- .nconfigs = 7,
+ .cfgs_offset = 0x2c,
.layout = {
.xstride = { 2 },
.default_color = 3,
.regs_offset = 0x140,
.id = 1,
.type = ATMEL_HLCDC_OVERLAY_LAYER,
- .nconfigs = 10,
+ .cfgs_offset = 0x2c,
.layout = {
.pos = 2,
.size = 3,
.regs_offset = 0x240,
.id = 2,
.type = ATMEL_HLCDC_OVERLAY_LAYER,
- .nconfigs = 10,
+ .cfgs_offset = 0x2c,
.layout = {
.pos = 2,
.size = 3,
.regs_offset = 0x340,
.id = 3,
.type = ATMEL_HLCDC_OVERLAY_LAYER,
- .nconfigs = 42,
+ .cfgs_offset = 0x4c,
.layout = {
.pos = 2,
.size = 3,
.chroma_key = 10,
.chroma_key_mask = 11,
.general_config = 12,
+ .scaler_config = 13,
+ .phicoeffs = {
+ .x = 17,
+ .y = 33,
+ },
.csc = 14,
},
},
return MODE_OK;
}
+static void atmel_hlcdc_layer_irq(struct atmel_hlcdc_layer *layer)
+{
+ if (!layer)
+ return;
+
+ if (layer->desc->type == ATMEL_HLCDC_BASE_LAYER ||
+ layer->desc->type == ATMEL_HLCDC_OVERLAY_LAYER ||
+ layer->desc->type == ATMEL_HLCDC_CURSOR_LAYER)
+ atmel_hlcdc_plane_irq(atmel_hlcdc_layer_to_plane(layer));
+}
+
static irqreturn_t atmel_hlcdc_dc_irq_handler(int irq, void *data)
{
struct drm_device *dev = data;
atmel_hlcdc_crtc_irq(dc->crtc);
for (i = 0; i < ATMEL_HLCDC_MAX_LAYERS; i++) {
- struct atmel_hlcdc_layer *layer = dc->layers[i];
-
- if (!(ATMEL_HLCDC_LAYER_STATUS(i) & status) || !layer)
- continue;
-
- atmel_hlcdc_layer_irq(layer);
+ if (ATMEL_HLCDC_LAYER_STATUS(i) & status)
+ atmel_hlcdc_layer_irq(dc->layers[i]);
}
return IRQ_HANDLED;
static int atmel_hlcdc_dc_modeset_init(struct drm_device *dev)
{
struct atmel_hlcdc_dc *dc = dev->dev_private;
- struct atmel_hlcdc_planes *planes;
int ret;
- int i;
drm_mode_config_init(dev);
return ret;
}
- planes = atmel_hlcdc_create_planes(dev);
- if (IS_ERR(planes)) {
- dev_err(dev->dev, "failed to create planes\n");
- return PTR_ERR(planes);
+ ret = atmel_hlcdc_create_planes(dev);
+ if (ret) {
+ dev_err(dev->dev, "failed to create planes: %d\n", ret);
+ return ret;
}
- dc->planes = planes;
-
- dc->layers[planes->primary->layer.desc->id] =
- &planes->primary->layer;
-
- if (planes->cursor)
- dc->layers[planes->cursor->layer.desc->id] =
- &planes->cursor->layer;
-
- for (i = 0; i < planes->noverlays; i++)
- dc->layers[planes->overlays[i]->layer.desc->id] =
- &planes->overlays[i]->layer;
-
ret = atmel_hlcdc_crtc_create(dev);
if (ret) {
dev_err(dev->dev, "failed to create crtc\n");
regmap_read(dc->hlcdc->regmap, ATMEL_HLCDC_ISR, &isr);
}
-static int atmel_hlcdc_dc_enable_vblank(struct drm_device *dev,
- unsigned int pipe)
-{
- struct atmel_hlcdc_dc *dc = dev->dev_private;
-
- /* Enable SOF (Start Of Frame) interrupt for vblank counting */
- regmap_write(dc->hlcdc->regmap, ATMEL_HLCDC_IER, ATMEL_HLCDC_SOF);
-
- return 0;
-}
-
-static void atmel_hlcdc_dc_disable_vblank(struct drm_device *dev,
- unsigned int pipe)
-{
- struct atmel_hlcdc_dc *dc = dev->dev_private;
-
- regmap_write(dc->hlcdc->regmap, ATMEL_HLCDC_IDR, ATMEL_HLCDC_SOF);
-}
-
-static const struct file_operations fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .poll = drm_poll,
- .read = drm_read,
- .llseek = no_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(fops);
static struct drm_driver atmel_hlcdc_dc_driver = {
.driver_features = DRIVER_HAVE_IRQ | DRIVER_GEM |
.irq_preinstall = atmel_hlcdc_dc_irq_uninstall,
.irq_postinstall = atmel_hlcdc_dc_irq_postinstall,
.irq_uninstall = atmel_hlcdc_dc_irq_uninstall,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = atmel_hlcdc_dc_enable_vblank,
- .disable_vblank = atmel_hlcdc_dc_disable_vblank,
.gem_free_object_unlocked = drm_gem_cma_free_object,
.gem_vm_ops = &drm_gem_cma_vm_ops,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
static int atmel_hlcdc_dc_drm_suspend(struct device *dev)
{
struct drm_device *drm_dev = dev_get_drvdata(dev);
- struct drm_crtc *crtc;
+ struct atmel_hlcdc_dc *dc = drm_dev->dev_private;
+ struct regmap *regmap = dc->hlcdc->regmap;
+ struct drm_atomic_state *state;
+
+ state = drm_atomic_helper_suspend(drm_dev);
+ if (IS_ERR(state))
+ return PTR_ERR(state);
- if (pm_runtime_suspended(dev))
- return 0;
+ dc->suspend.state = state;
+
+ regmap_read(regmap, ATMEL_HLCDC_IMR, &dc->suspend.imr);
+ regmap_write(regmap, ATMEL_HLCDC_IDR, dc->suspend.imr);
+ clk_disable_unprepare(dc->hlcdc->periph_clk);
- drm_modeset_lock_all(drm_dev);
- list_for_each_entry(crtc, &drm_dev->mode_config.crtc_list, head)
- atmel_hlcdc_crtc_suspend(crtc);
- drm_modeset_unlock_all(drm_dev);
return 0;
}
static int atmel_hlcdc_dc_drm_resume(struct device *dev)
{
struct drm_device *drm_dev = dev_get_drvdata(dev);
- struct drm_crtc *crtc;
+ struct atmel_hlcdc_dc *dc = drm_dev->dev_private;
- if (pm_runtime_suspended(dev))
- return 0;
+ clk_prepare_enable(dc->hlcdc->periph_clk);
+ regmap_write(dc->hlcdc->regmap, ATMEL_HLCDC_IER, dc->suspend.imr);
- drm_modeset_lock_all(drm_dev);
- list_for_each_entry(crtc, &drm_dev->mode_config.crtc_list, head)
- atmel_hlcdc_crtc_resume(crtc);
- drm_modeset_unlock_all(drm_dev);
- return 0;
+ return drm_atomic_helper_resume(drm_dev, dc->suspend.state);
}
#endif
#define DRM_ATMEL_HLCDC_H
#include <linux/clk.h>
+#include <linux/dmapool.h>
#include <linux/irqdomain.h>
+#include <linux/mfd/atmel-hlcdc.h>
#include <linux/pwm.h>
#include <drm/drm_atomic.h>
#include <drm/drm_plane_helper.h>
#include <drm/drmP.h>
-#include "atmel_hlcdc_layer.h"
+#define ATMEL_HLCDC_LAYER_CHER 0x0
+#define ATMEL_HLCDC_LAYER_CHDR 0x4
+#define ATMEL_HLCDC_LAYER_CHSR 0x8
+#define ATMEL_HLCDC_LAYER_EN BIT(0)
+#define ATMEL_HLCDC_LAYER_UPDATE BIT(1)
+#define ATMEL_HLCDC_LAYER_A2Q BIT(2)
+#define ATMEL_HLCDC_LAYER_RST BIT(8)
-#define ATMEL_HLCDC_MAX_LAYERS 5
+#define ATMEL_HLCDC_LAYER_IER 0xc
+#define ATMEL_HLCDC_LAYER_IDR 0x10
+#define ATMEL_HLCDC_LAYER_IMR 0x14
+#define ATMEL_HLCDC_LAYER_ISR 0x18
+#define ATMEL_HLCDC_LAYER_DFETCH BIT(0)
+#define ATMEL_HLCDC_LAYER_LFETCH BIT(1)
+#define ATMEL_HLCDC_LAYER_DMA_IRQ(p) BIT(2 + (8 * (p)))
+#define ATMEL_HLCDC_LAYER_DSCR_IRQ(p) BIT(3 + (8 * (p)))
+#define ATMEL_HLCDC_LAYER_ADD_IRQ(p) BIT(4 + (8 * (p)))
+#define ATMEL_HLCDC_LAYER_DONE_IRQ(p) BIT(5 + (8 * (p)))
+#define ATMEL_HLCDC_LAYER_OVR_IRQ(p) BIT(6 + (8 * (p)))
+
+#define ATMEL_HLCDC_LAYER_PLANE_HEAD(p) (((p) * 0x10) + 0x1c)
+#define ATMEL_HLCDC_LAYER_PLANE_ADDR(p) (((p) * 0x10) + 0x20)
+#define ATMEL_HLCDC_LAYER_PLANE_CTRL(p) (((p) * 0x10) + 0x24)
+#define ATMEL_HLCDC_LAYER_PLANE_NEXT(p) (((p) * 0x10) + 0x28)
+
+#define ATMEL_HLCDC_LAYER_DMA_CFG 0
+#define ATMEL_HLCDC_LAYER_DMA_SIF BIT(0)
+#define ATMEL_HLCDC_LAYER_DMA_BLEN_MASK GENMASK(5, 4)
+#define ATMEL_HLCDC_LAYER_DMA_BLEN_SINGLE (0 << 4)
+#define ATMEL_HLCDC_LAYER_DMA_BLEN_INCR4 (1 << 4)
+#define ATMEL_HLCDC_LAYER_DMA_BLEN_INCR8 (2 << 4)
+#define ATMEL_HLCDC_LAYER_DMA_BLEN_INCR16 (3 << 4)
+#define ATMEL_HLCDC_LAYER_DMA_DLBO BIT(8)
+#define ATMEL_HLCDC_LAYER_DMA_ROTDIS BIT(12)
+#define ATMEL_HLCDC_LAYER_DMA_LOCKDIS BIT(13)
+
+#define ATMEL_HLCDC_LAYER_FORMAT_CFG 1
+#define ATMEL_HLCDC_LAYER_RGB (0 << 0)
+#define ATMEL_HLCDC_LAYER_CLUT (1 << 0)
+#define ATMEL_HLCDC_LAYER_YUV (2 << 0)
+#define ATMEL_HLCDC_RGB_MODE(m) \
+ (ATMEL_HLCDC_LAYER_RGB | (((m) & 0xf) << 4))
+#define ATMEL_HLCDC_CLUT_MODE(m) \
+ (ATMEL_HLCDC_LAYER_CLUT | (((m) & 0x3) << 8))
+#define ATMEL_HLCDC_YUV_MODE(m) \
+ (ATMEL_HLCDC_LAYER_YUV | (((m) & 0xf) << 12))
+#define ATMEL_HLCDC_YUV422ROT BIT(16)
+#define ATMEL_HLCDC_YUV422SWP BIT(17)
+#define ATMEL_HLCDC_DSCALEOPT BIT(20)
+
+#define ATMEL_HLCDC_XRGB4444_MODE ATMEL_HLCDC_RGB_MODE(0)
+#define ATMEL_HLCDC_ARGB4444_MODE ATMEL_HLCDC_RGB_MODE(1)
+#define ATMEL_HLCDC_RGBA4444_MODE ATMEL_HLCDC_RGB_MODE(2)
+#define ATMEL_HLCDC_RGB565_MODE ATMEL_HLCDC_RGB_MODE(3)
+#define ATMEL_HLCDC_ARGB1555_MODE ATMEL_HLCDC_RGB_MODE(4)
+#define ATMEL_HLCDC_XRGB8888_MODE ATMEL_HLCDC_RGB_MODE(9)
+#define ATMEL_HLCDC_RGB888_MODE ATMEL_HLCDC_RGB_MODE(10)
+#define ATMEL_HLCDC_ARGB8888_MODE ATMEL_HLCDC_RGB_MODE(12)
+#define ATMEL_HLCDC_RGBA8888_MODE ATMEL_HLCDC_RGB_MODE(13)
+
+#define ATMEL_HLCDC_AYUV_MODE ATMEL_HLCDC_YUV_MODE(0)
+#define ATMEL_HLCDC_YUYV_MODE ATMEL_HLCDC_YUV_MODE(1)
+#define ATMEL_HLCDC_UYVY_MODE ATMEL_HLCDC_YUV_MODE(2)
+#define ATMEL_HLCDC_YVYU_MODE ATMEL_HLCDC_YUV_MODE(3)
+#define ATMEL_HLCDC_VYUY_MODE ATMEL_HLCDC_YUV_MODE(4)
+#define ATMEL_HLCDC_NV61_MODE ATMEL_HLCDC_YUV_MODE(5)
+#define ATMEL_HLCDC_YUV422_MODE ATMEL_HLCDC_YUV_MODE(6)
+#define ATMEL_HLCDC_NV21_MODE ATMEL_HLCDC_YUV_MODE(7)
+#define ATMEL_HLCDC_YUV420_MODE ATMEL_HLCDC_YUV_MODE(8)
+
+#define ATMEL_HLCDC_LAYER_POS(x, y) ((x) | ((y) << 16))
+#define ATMEL_HLCDC_LAYER_SIZE(w, h) (((w) - 1) | (((h) - 1) << 16))
+
+#define ATMEL_HLCDC_LAYER_CRKEY BIT(0)
+#define ATMEL_HLCDC_LAYER_INV BIT(1)
+#define ATMEL_HLCDC_LAYER_ITER2BL BIT(2)
+#define ATMEL_HLCDC_LAYER_ITER BIT(3)
+#define ATMEL_HLCDC_LAYER_REVALPHA BIT(4)
+#define ATMEL_HLCDC_LAYER_GAEN BIT(5)
+#define ATMEL_HLCDC_LAYER_LAEN BIT(6)
+#define ATMEL_HLCDC_LAYER_OVR BIT(7)
+#define ATMEL_HLCDC_LAYER_DMA BIT(8)
+#define ATMEL_HLCDC_LAYER_REP BIT(9)
+#define ATMEL_HLCDC_LAYER_DSTKEY BIT(10)
+#define ATMEL_HLCDC_LAYER_DISCEN BIT(11)
+#define ATMEL_HLCDC_LAYER_GA_SHIFT 16
+#define ATMEL_HLCDC_LAYER_GA_MASK \
+ GENMASK(23, ATMEL_HLCDC_LAYER_GA_SHIFT)
+#define ATMEL_HLCDC_LAYER_GA(x) \
+ ((x) << ATMEL_HLCDC_LAYER_GA_SHIFT)
+
+#define ATMEL_HLCDC_LAYER_DISC_POS(x, y) ((x) | ((y) << 16))
+#define ATMEL_HLCDC_LAYER_DISC_SIZE(w, h) (((w) - 1) | (((h) - 1) << 16))
+
+#define ATMEL_HLCDC_LAYER_SCALER_FACTORS(x, y) ((x) | ((y) << 16))
+#define ATMEL_HLCDC_LAYER_SCALER_ENABLE BIT(31)
+
+#define ATMEL_HLCDC_LAYER_MAX_PLANES 3
+
+#define ATMEL_HLCDC_DMA_CHANNEL_DSCR_RESERVED BIT(0)
+#define ATMEL_HLCDC_DMA_CHANNEL_DSCR_LOADED BIT(1)
+#define ATMEL_HLCDC_DMA_CHANNEL_DSCR_DONE BIT(2)
+#define ATMEL_HLCDC_DMA_CHANNEL_DSCR_OVERRUN BIT(3)
+
+#define ATMEL_HLCDC_MAX_LAYERS 6
/**
- * Atmel HLCDC Display Controller description structure.
+ * Atmel HLCDC Layer registers layout structure
*
- * This structure describe the HLCDC IP capabilities and depends on the
- * HLCDC IP version (or Atmel SoC family).
+ * Each HLCDC layer has its own register organization and a given register
+ * can be placed differently on 2 different layers depending on its
+ * capabilities.
+ * This structure stores common registers layout for a given layer and is
+ * used by HLCDC layer code to choose the appropriate register to write to
+ * or to read from.
*
- * @min_width: minimum width supported by the Display Controller
- * @min_height: minimum height supported by the Display Controller
- * @max_width: maximum width supported by the Display Controller
- * @max_height: maximum height supported by the Display Controller
- * @max_spw: maximum vertical/horizontal pulse width
- * @max_vpw: maximum vertical back/front porch width
- * @max_hpw: maximum horizontal back/front porch width
- * @conflicting_output_formats: true if RGBXXX output formats conflict with
- * each other.
- * @layers: a layer description table describing available layers
- * @nlayers: layer description table size
+ * For all fields, a value of zero means "unsupported".
+ *
+ * See Atmel's datasheet for a detailled description of these registers.
+ *
+ * @xstride: xstride registers
+ * @pstride: pstride registers
+ * @pos: position register
+ * @size: displayed size register
+ * @memsize: memory size register
+ * @default_color: default color register
+ * @chroma_key: chroma key register
+ * @chroma_key_mask: chroma key mask register
+ * @general_config: general layer config register
+ * @sacler_config: scaler factors register
+ * @phicoeffs: X/Y PHI coefficient registers
+ * @disc_pos: discard area position register
+ * @disc_size: discard area size register
+ * @csc: color space conversion register
*/
-struct atmel_hlcdc_dc_desc {
- int min_width;
- int min_height;
+struct atmel_hlcdc_layer_cfg_layout {
+ int xstride[ATMEL_HLCDC_LAYER_MAX_PLANES];
+ int pstride[ATMEL_HLCDC_LAYER_MAX_PLANES];
+ int pos;
+ int size;
+ int memsize;
+ int default_color;
+ int chroma_key;
+ int chroma_key_mask;
+ int general_config;
+ int scaler_config;
+ struct {
+ int x;
+ int y;
+ } phicoeffs;
+ int disc_pos;
+ int disc_size;
+ int csc;
+};
+
+/**
+ * Atmel HLCDC DMA descriptor structure
+ *
+ * This structure is used by the HLCDC DMA engine to schedule a DMA transfer.
+ *
+ * The structure fields must remain in this specific order, because they're
+ * used by the HLCDC DMA engine, which expect them in this order.
+ * HLCDC DMA descriptors must be aligned on 64 bits.
+ *
+ * @addr: buffer DMA address
+ * @ctrl: DMA transfer options
+ * @next: next DMA descriptor to fetch
+ * @self: descriptor DMA address
+ */
+struct atmel_hlcdc_dma_channel_dscr {
+ dma_addr_t addr;
+ u32 ctrl;
+ dma_addr_t next;
+ dma_addr_t self;
+} __aligned(sizeof(u64));
+
+/**
+ * Atmel HLCDC layer types
+ */
+enum atmel_hlcdc_layer_type {
+ ATMEL_HLCDC_NO_LAYER,
+ ATMEL_HLCDC_BASE_LAYER,
+ ATMEL_HLCDC_OVERLAY_LAYER,
+ ATMEL_HLCDC_CURSOR_LAYER,
+ ATMEL_HLCDC_PP_LAYER,
+};
+
+/**
+ * Atmel HLCDC Supported formats structure
+ *
+ * This structure list all the formats supported by a given layer.
+ *
+ * @nformats: number of supported formats
+ * @formats: supported formats
+ */
+struct atmel_hlcdc_formats {
+ int nformats;
+ u32 *formats;
+};
+
+/**
+ * Atmel HLCDC Layer description structure
+ *
+ * This structure describes the capabilities provided by a given layer.
+ *
+ * @name: layer name
+ * @type: layer type
+ * @id: layer id
+ * @regs_offset: offset of the layer registers from the HLCDC registers base
+ * @cfgs_offset: CFGX registers offset from the layer registers base
+ * @formats: supported formats
+ * @layout: config registers layout
+ * @max_width: maximum width supported by this layer (0 means unlimited)
+ * @max_height: maximum height supported by this layer (0 means unlimited)
+ */
+struct atmel_hlcdc_layer_desc {
+ const char *name;
+ enum atmel_hlcdc_layer_type type;
+ int id;
+ int regs_offset;
+ int cfgs_offset;
+ struct atmel_hlcdc_formats *formats;
+ struct atmel_hlcdc_layer_cfg_layout layout;
int max_width;
int max_height;
- int max_spw;
- int max_vpw;
- int max_hpw;
- bool conflicting_output_formats;
- const struct atmel_hlcdc_layer_desc *layers;
- int nlayers;
};
/**
- * Atmel HLCDC Plane properties.
+ * Atmel HLCDC Layer.
*
- * This structure stores plane property definitions.
+ * A layer can be a DRM plane of a post processing layer used to render
+ * HLCDC composition into memory.
*
- * @alpha: alpha blending (or transparency) property
- * @rotation: rotation property
+ * @desc: layer description
+ * @regmap: pointer to the HLCDC regmap
*/
-struct atmel_hlcdc_plane_properties {
- struct drm_property *alpha;
+struct atmel_hlcdc_layer {
+ const struct atmel_hlcdc_layer_desc *desc;
+ struct regmap *regmap;
};
/**
* @base: base DRM plane structure
* @layer: HLCDC layer structure
* @properties: pointer to the property definitions structure
- * @rotation: current rotation status
*/
struct atmel_hlcdc_plane {
struct drm_plane base;
}
static inline struct atmel_hlcdc_plane *
-atmel_hlcdc_layer_to_plane(struct atmel_hlcdc_layer *l)
+atmel_hlcdc_layer_to_plane(struct atmel_hlcdc_layer *layer)
{
- return container_of(l, struct atmel_hlcdc_plane, layer);
+ return container_of(layer, struct atmel_hlcdc_plane, layer);
}
/**
- * Atmel HLCDC Planes.
+ * Atmel HLCDC Display Controller description structure.
+ *
+ * This structure describes the HLCDC IP capabilities and depends on the
+ * HLCDC IP version (or Atmel SoC family).
+ *
+ * @min_width: minimum width supported by the Display Controller
+ * @min_height: minimum height supported by the Display Controller
+ * @max_width: maximum width supported by the Display Controller
+ * @max_height: maximum height supported by the Display Controller
+ * @max_spw: maximum vertical/horizontal pulse width
+ * @max_vpw: maximum vertical back/front porch width
+ * @max_hpw: maximum horizontal back/front porch width
+ * @conflicting_output_formats: true if RGBXXX output formats conflict with
+ * each other.
+ * @layers: a layer description table describing available layers
+ * @nlayers: layer description table size
+ */
+struct atmel_hlcdc_dc_desc {
+ int min_width;
+ int min_height;
+ int max_width;
+ int max_height;
+ int max_spw;
+ int max_vpw;
+ int max_hpw;
+ bool conflicting_output_formats;
+ const struct atmel_hlcdc_layer_desc *layers;
+ int nlayers;
+};
+
+/**
+ * Atmel HLCDC Plane properties.
*
- * This structure stores the instantiated HLCDC Planes and can be accessed by
- * the HLCDC Display Controller or the HLCDC CRTC.
+ * This structure stores plane property definitions.
*
- * @primary: primary plane
- * @cursor: hardware cursor plane
- * @overlays: overlay plane table
- * @noverlays: number of overlay planes
+ * @alpha: alpha blending (or transparency) property
+ * @rotation: rotation property
*/
-struct atmel_hlcdc_planes {
- struct atmel_hlcdc_plane *primary;
- struct atmel_hlcdc_plane *cursor;
- struct atmel_hlcdc_plane **overlays;
- int noverlays;
+struct atmel_hlcdc_plane_properties {
+ struct drm_property *alpha;
};
/**
* Atmel HLCDC Display Controller.
*
* @desc: HLCDC Display Controller description
+ * @dscrpool: DMA coherent pool used to allocate DMA descriptors
* @hlcdc: pointer to the atmel_hlcdc structure provided by the MFD device
* @fbdev: framebuffer device attached to the Display Controller
* @crtc: CRTC provided by the display controller
* @planes: instantiated planes
- * @layers: active HLCDC layer
+ * @layers: active HLCDC layers
* @wq: display controller workqueue
+ * @suspend: used to store the HLCDC state when entering suspend
* @commit: used for async commit handling
*/
struct atmel_hlcdc_dc {
const struct atmel_hlcdc_dc_desc *desc;
+ struct dma_pool *dscrpool;
struct atmel_hlcdc *hlcdc;
struct drm_fbdev_cma *fbdev;
struct drm_crtc *crtc;
- struct atmel_hlcdc_planes *planes;
struct atmel_hlcdc_layer *layers[ATMEL_HLCDC_MAX_LAYERS];
struct workqueue_struct *wq;
+ struct {
+ u32 imr;
+ struct drm_atomic_state *state;
+ } suspend;
struct {
wait_queue_head_t wait;
bool pending;
extern struct atmel_hlcdc_formats atmel_hlcdc_plane_rgb_formats;
extern struct atmel_hlcdc_formats atmel_hlcdc_plane_rgb_and_yuv_formats;
+static inline void atmel_hlcdc_layer_write_reg(struct atmel_hlcdc_layer *layer,
+ unsigned int reg, u32 val)
+{
+ regmap_write(layer->regmap, layer->desc->regs_offset + reg, val);
+}
+
+static inline u32 atmel_hlcdc_layer_read_reg(struct atmel_hlcdc_layer *layer,
+ unsigned int reg)
+{
+ u32 val;
+
+ regmap_read(layer->regmap, layer->desc->regs_offset + reg, &val);
+
+ return val;
+}
+
+static inline void atmel_hlcdc_layer_write_cfg(struct atmel_hlcdc_layer *layer,
+ unsigned int cfgid, u32 val)
+{
+ atmel_hlcdc_layer_write_reg(layer,
+ layer->desc->cfgs_offset +
+ (cfgid * sizeof(u32)), val);
+}
+
+static inline u32 atmel_hlcdc_layer_read_cfg(struct atmel_hlcdc_layer *layer,
+ unsigned int cfgid)
+{
+ return atmel_hlcdc_layer_read_reg(layer,
+ layer->desc->cfgs_offset +
+ (cfgid * sizeof(u32)));
+}
+
+static inline void atmel_hlcdc_layer_init(struct atmel_hlcdc_layer *layer,
+ const struct atmel_hlcdc_layer_desc *desc,
+ struct regmap *regmap)
+{
+ layer->desc = desc;
+ layer->regmap = regmap;
+}
+
int atmel_hlcdc_dc_mode_valid(struct atmel_hlcdc_dc *dc,
struct drm_display_mode *mode);
-struct atmel_hlcdc_planes *
-atmel_hlcdc_create_planes(struct drm_device *dev);
+int atmel_hlcdc_create_planes(struct drm_device *dev);
+void atmel_hlcdc_plane_irq(struct atmel_hlcdc_plane *plane);
int atmel_hlcdc_plane_prepare_disc_area(struct drm_crtc_state *c_state);
int atmel_hlcdc_plane_prepare_ahb_routing(struct drm_crtc_state *c_state);
void atmel_hlcdc_crtc_irq(struct drm_crtc *c);
-void atmel_hlcdc_crtc_suspend(struct drm_crtc *crtc);
-void atmel_hlcdc_crtc_resume(struct drm_crtc *crtc);
-
int atmel_hlcdc_crtc_create(struct drm_device *dev);
int atmel_hlcdc_create_outputs(struct drm_device *dev);
+++ /dev/null
-/*
- * Copyright (C) 2014 Free Electrons
- * Copyright (C) 2014 Atmel
- *
- * Author: Boris BREZILLON <boris.brezillon@free-electrons.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License version 2 as published by
- * the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- *
- * You should have received a copy of the GNU General Public License along with
- * this program. If not, see <http://www.gnu.org/licenses/>.
- */
-
-#include <linux/dma-mapping.h>
-#include <linux/interrupt.h>
-
-#include "atmel_hlcdc_dc.h"
-
-static void
-atmel_hlcdc_layer_fb_flip_release(struct drm_flip_work *work, void *val)
-{
- struct atmel_hlcdc_layer_fb_flip *flip = val;
-
- if (flip->fb)
- drm_framebuffer_unreference(flip->fb);
- kfree(flip);
-}
-
-static void
-atmel_hlcdc_layer_fb_flip_destroy(struct atmel_hlcdc_layer_fb_flip *flip)
-{
- if (flip->fb)
- drm_framebuffer_unreference(flip->fb);
- kfree(flip->task);
- kfree(flip);
-}
-
-static void
-atmel_hlcdc_layer_fb_flip_release_queue(struct atmel_hlcdc_layer *layer,
- struct atmel_hlcdc_layer_fb_flip *flip)
-{
- int i;
-
- if (!flip)
- return;
-
- for (i = 0; i < layer->max_planes; i++) {
- if (!flip->dscrs[i])
- break;
-
- flip->dscrs[i]->status = 0;
- flip->dscrs[i] = NULL;
- }
-
- drm_flip_work_queue_task(&layer->gc, flip->task);
- drm_flip_work_commit(&layer->gc, layer->wq);
-}
-
-static void atmel_hlcdc_layer_update_reset(struct atmel_hlcdc_layer *layer,
- int id)
-{
- struct atmel_hlcdc_layer_update *upd = &layer->update;
- struct atmel_hlcdc_layer_update_slot *slot;
-
- if (id < 0 || id > 1)
- return;
-
- slot = &upd->slots[id];
- bitmap_clear(slot->updated_configs, 0, layer->desc->nconfigs);
- memset(slot->configs, 0,
- sizeof(*slot->configs) * layer->desc->nconfigs);
-
- if (slot->fb_flip) {
- atmel_hlcdc_layer_fb_flip_release_queue(layer, slot->fb_flip);
- slot->fb_flip = NULL;
- }
-}
-
-static void atmel_hlcdc_layer_update_apply(struct atmel_hlcdc_layer *layer)
-{
- struct atmel_hlcdc_layer_dma_channel *dma = &layer->dma;
- const struct atmel_hlcdc_layer_desc *desc = layer->desc;
- struct atmel_hlcdc_layer_update *upd = &layer->update;
- struct regmap *regmap = layer->hlcdc->regmap;
- struct atmel_hlcdc_layer_update_slot *slot;
- struct atmel_hlcdc_layer_fb_flip *fb_flip;
- struct atmel_hlcdc_dma_channel_dscr *dscr;
- unsigned int cfg;
- u32 action = 0;
- int i = 0;
-
- if (upd->pending < 0 || upd->pending > 1)
- return;
-
- slot = &upd->slots[upd->pending];
-
- for_each_set_bit(cfg, slot->updated_configs, layer->desc->nconfigs) {
- regmap_write(regmap,
- desc->regs_offset +
- ATMEL_HLCDC_LAYER_CFG(layer, cfg),
- slot->configs[cfg]);
- action |= ATMEL_HLCDC_LAYER_UPDATE;
- }
-
- fb_flip = slot->fb_flip;
-
- if (!fb_flip->fb)
- goto apply;
-
- if (dma->status == ATMEL_HLCDC_LAYER_DISABLED) {
- for (i = 0; i < fb_flip->ngems; i++) {
- dscr = fb_flip->dscrs[i];
- dscr->ctrl = ATMEL_HLCDC_LAYER_DFETCH |
- ATMEL_HLCDC_LAYER_DMA_IRQ |
- ATMEL_HLCDC_LAYER_ADD_IRQ |
- ATMEL_HLCDC_LAYER_DONE_IRQ;
-
- regmap_write(regmap,
- desc->regs_offset +
- ATMEL_HLCDC_LAYER_PLANE_ADDR(i),
- dscr->addr);
- regmap_write(regmap,
- desc->regs_offset +
- ATMEL_HLCDC_LAYER_PLANE_CTRL(i),
- dscr->ctrl);
- regmap_write(regmap,
- desc->regs_offset +
- ATMEL_HLCDC_LAYER_PLANE_NEXT(i),
- dscr->next);
- }
-
- action |= ATMEL_HLCDC_LAYER_DMA_CHAN;
- dma->status = ATMEL_HLCDC_LAYER_ENABLED;
- } else {
- for (i = 0; i < fb_flip->ngems; i++) {
- dscr = fb_flip->dscrs[i];
- dscr->ctrl = ATMEL_HLCDC_LAYER_DFETCH |
- ATMEL_HLCDC_LAYER_DMA_IRQ |
- ATMEL_HLCDC_LAYER_DSCR_IRQ |
- ATMEL_HLCDC_LAYER_DONE_IRQ;
-
- regmap_write(regmap,
- desc->regs_offset +
- ATMEL_HLCDC_LAYER_PLANE_HEAD(i),
- dscr->next);
- }
-
- action |= ATMEL_HLCDC_LAYER_A2Q;
- }
-
- /* Release unneeded descriptors */
- for (i = fb_flip->ngems; i < layer->max_planes; i++) {
- fb_flip->dscrs[i]->status = 0;
- fb_flip->dscrs[i] = NULL;
- }
-
- dma->queue = fb_flip;
- slot->fb_flip = NULL;
-
-apply:
- if (action)
- regmap_write(regmap,
- desc->regs_offset + ATMEL_HLCDC_LAYER_CHER,
- action);
-
- atmel_hlcdc_layer_update_reset(layer, upd->pending);
-
- upd->pending = -1;
-}
-
-void atmel_hlcdc_layer_irq(struct atmel_hlcdc_layer *layer)
-{
- struct atmel_hlcdc_layer_dma_channel *dma = &layer->dma;
- const struct atmel_hlcdc_layer_desc *desc = layer->desc;
- struct regmap *regmap = layer->hlcdc->regmap;
- struct atmel_hlcdc_layer_fb_flip *flip;
- unsigned long flags;
- unsigned int isr, imr;
- unsigned int status;
- unsigned int plane_status;
- u32 flip_status;
-
- int i;
-
- regmap_read(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_IMR, &imr);
- regmap_read(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_ISR, &isr);
- status = imr & isr;
- if (!status)
- return;
-
- spin_lock_irqsave(&layer->lock, flags);
-
- flip = dma->queue ? dma->queue : dma->cur;
-
- if (!flip) {
- spin_unlock_irqrestore(&layer->lock, flags);
- return;
- }
-
- /*
- * Set LOADED and DONE flags: they'll be cleared if at least one
- * memory plane is not LOADED or DONE.
- */
- flip_status = ATMEL_HLCDC_DMA_CHANNEL_DSCR_LOADED |
- ATMEL_HLCDC_DMA_CHANNEL_DSCR_DONE;
- for (i = 0; i < flip->ngems; i++) {
- plane_status = (status >> (8 * i));
-
- if (plane_status &
- (ATMEL_HLCDC_LAYER_ADD_IRQ |
- ATMEL_HLCDC_LAYER_DSCR_IRQ) &
- ~flip->dscrs[i]->ctrl) {
- flip->dscrs[i]->status |=
- ATMEL_HLCDC_DMA_CHANNEL_DSCR_LOADED;
- flip->dscrs[i]->ctrl |=
- ATMEL_HLCDC_LAYER_ADD_IRQ |
- ATMEL_HLCDC_LAYER_DSCR_IRQ;
- }
-
- if (plane_status &
- ATMEL_HLCDC_LAYER_DONE_IRQ &
- ~flip->dscrs[i]->ctrl) {
- flip->dscrs[i]->status |=
- ATMEL_HLCDC_DMA_CHANNEL_DSCR_DONE;
- flip->dscrs[i]->ctrl |=
- ATMEL_HLCDC_LAYER_DONE_IRQ;
- }
-
- if (plane_status & ATMEL_HLCDC_LAYER_OVR_IRQ)
- flip->dscrs[i]->status |=
- ATMEL_HLCDC_DMA_CHANNEL_DSCR_OVERRUN;
-
- /*
- * Clear LOADED and DONE flags if the memory plane is either
- * not LOADED or not DONE.
- */
- if (!(flip->dscrs[i]->status &
- ATMEL_HLCDC_DMA_CHANNEL_DSCR_LOADED))
- flip_status &= ~ATMEL_HLCDC_DMA_CHANNEL_DSCR_LOADED;
-
- if (!(flip->dscrs[i]->status &
- ATMEL_HLCDC_DMA_CHANNEL_DSCR_DONE))
- flip_status &= ~ATMEL_HLCDC_DMA_CHANNEL_DSCR_DONE;
-
- /*
- * An overrun on one memory plane impact the whole framebuffer
- * transfer, hence we set the OVERRUN flag as soon as there's
- * one memory plane reporting such an overrun.
- */
- flip_status |= flip->dscrs[i]->status &
- ATMEL_HLCDC_DMA_CHANNEL_DSCR_OVERRUN;
- }
-
- /* Get changed bits */
- flip_status ^= flip->status;
- flip->status |= flip_status;
-
- if (flip_status & ATMEL_HLCDC_DMA_CHANNEL_DSCR_LOADED) {
- atmel_hlcdc_layer_fb_flip_release_queue(layer, dma->cur);
- dma->cur = dma->queue;
- dma->queue = NULL;
- }
-
- if (flip_status & ATMEL_HLCDC_DMA_CHANNEL_DSCR_DONE) {
- atmel_hlcdc_layer_fb_flip_release_queue(layer, dma->cur);
- dma->cur = NULL;
- }
-
- if (flip_status & ATMEL_HLCDC_DMA_CHANNEL_DSCR_OVERRUN) {
- regmap_write(regmap,
- desc->regs_offset + ATMEL_HLCDC_LAYER_CHDR,
- ATMEL_HLCDC_LAYER_RST);
- if (dma->queue)
- atmel_hlcdc_layer_fb_flip_release_queue(layer,
- dma->queue);
-
- if (dma->cur)
- atmel_hlcdc_layer_fb_flip_release_queue(layer,
- dma->cur);
-
- dma->cur = NULL;
- dma->queue = NULL;
- }
-
- if (!dma->queue) {
- atmel_hlcdc_layer_update_apply(layer);
-
- if (!dma->cur)
- dma->status = ATMEL_HLCDC_LAYER_DISABLED;
- }
-
- spin_unlock_irqrestore(&layer->lock, flags);
-}
-
-void atmel_hlcdc_layer_disable(struct atmel_hlcdc_layer *layer)
-{
- struct atmel_hlcdc_layer_dma_channel *dma = &layer->dma;
- struct atmel_hlcdc_layer_update *upd = &layer->update;
- struct regmap *regmap = layer->hlcdc->regmap;
- const struct atmel_hlcdc_layer_desc *desc = layer->desc;
- unsigned long flags;
- unsigned int isr;
-
- spin_lock_irqsave(&layer->lock, flags);
-
- /* Disable the layer */
- regmap_write(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_CHDR,
- ATMEL_HLCDC_LAYER_RST | ATMEL_HLCDC_LAYER_A2Q |
- ATMEL_HLCDC_LAYER_UPDATE);
-
- /* Clear all pending interrupts */
- regmap_read(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_ISR, &isr);
-
- /* Discard current and queued framebuffer transfers. */
- if (dma->cur) {
- atmel_hlcdc_layer_fb_flip_release_queue(layer, dma->cur);
- dma->cur = NULL;
- }
-
- if (dma->queue) {
- atmel_hlcdc_layer_fb_flip_release_queue(layer, dma->queue);
- dma->queue = NULL;
- }
-
- /*
- * Then discard the pending update request (if any) to prevent
- * DMA irq handler from restarting the DMA channel after it has
- * been disabled.
- */
- if (upd->pending >= 0) {
- atmel_hlcdc_layer_update_reset(layer, upd->pending);
- upd->pending = -1;
- }
-
- dma->status = ATMEL_HLCDC_LAYER_DISABLED;
-
- spin_unlock_irqrestore(&layer->lock, flags);
-}
-
-int atmel_hlcdc_layer_update_start(struct atmel_hlcdc_layer *layer)
-{
- struct atmel_hlcdc_layer_dma_channel *dma = &layer->dma;
- struct atmel_hlcdc_layer_update *upd = &layer->update;
- struct regmap *regmap = layer->hlcdc->regmap;
- struct atmel_hlcdc_layer_fb_flip *fb_flip;
- struct atmel_hlcdc_layer_update_slot *slot;
- unsigned long flags;
- int i, j = 0;
-
- fb_flip = kzalloc(sizeof(*fb_flip), GFP_KERNEL);
- if (!fb_flip)
- return -ENOMEM;
-
- fb_flip->task = drm_flip_work_allocate_task(fb_flip, GFP_KERNEL);
- if (!fb_flip->task) {
- kfree(fb_flip);
- return -ENOMEM;
- }
-
- spin_lock_irqsave(&layer->lock, flags);
-
- upd->next = upd->pending ? 0 : 1;
-
- slot = &upd->slots[upd->next];
-
- for (i = 0; i < layer->max_planes * 4; i++) {
- if (!dma->dscrs[i].status) {
- fb_flip->dscrs[j++] = &dma->dscrs[i];
- dma->dscrs[i].status =
- ATMEL_HLCDC_DMA_CHANNEL_DSCR_RESERVED;
- if (j == layer->max_planes)
- break;
- }
- }
-
- if (j < layer->max_planes) {
- for (i = 0; i < j; i++)
- fb_flip->dscrs[i]->status = 0;
- }
-
- if (j < layer->max_planes) {
- spin_unlock_irqrestore(&layer->lock, flags);
- atmel_hlcdc_layer_fb_flip_destroy(fb_flip);
- return -EBUSY;
- }
-
- slot->fb_flip = fb_flip;
-
- if (upd->pending >= 0) {
- memcpy(slot->configs,
- upd->slots[upd->pending].configs,
- layer->desc->nconfigs * sizeof(u32));
- memcpy(slot->updated_configs,
- upd->slots[upd->pending].updated_configs,
- DIV_ROUND_UP(layer->desc->nconfigs,
- BITS_PER_BYTE * sizeof(unsigned long)) *
- sizeof(unsigned long));
- slot->fb_flip->fb = upd->slots[upd->pending].fb_flip->fb;
- if (upd->slots[upd->pending].fb_flip->fb) {
- slot->fb_flip->fb =
- upd->slots[upd->pending].fb_flip->fb;
- slot->fb_flip->ngems =
- upd->slots[upd->pending].fb_flip->ngems;
- drm_framebuffer_reference(slot->fb_flip->fb);
- }
- } else {
- regmap_bulk_read(regmap,
- layer->desc->regs_offset +
- ATMEL_HLCDC_LAYER_CFG(layer, 0),
- upd->slots[upd->next].configs,
- layer->desc->nconfigs);
- }
-
- spin_unlock_irqrestore(&layer->lock, flags);
-
- return 0;
-}
-
-void atmel_hlcdc_layer_update_rollback(struct atmel_hlcdc_layer *layer)
-{
- struct atmel_hlcdc_layer_update *upd = &layer->update;
-
- atmel_hlcdc_layer_update_reset(layer, upd->next);
- upd->next = -1;
-}
-
-void atmel_hlcdc_layer_update_set_fb(struct atmel_hlcdc_layer *layer,
- struct drm_framebuffer *fb,
- unsigned int *offsets)
-{
- struct atmel_hlcdc_layer_update *upd = &layer->update;
- struct atmel_hlcdc_layer_fb_flip *fb_flip;
- struct atmel_hlcdc_layer_update_slot *slot;
- struct atmel_hlcdc_dma_channel_dscr *dscr;
- struct drm_framebuffer *old_fb;
- int nplanes = 0;
- int i;
-
- if (upd->next < 0 || upd->next > 1)
- return;
-
- if (fb)
- nplanes = fb->format->num_planes;
-
- if (nplanes > layer->max_planes)
- return;
-
- slot = &upd->slots[upd->next];
-
- fb_flip = slot->fb_flip;
- old_fb = slot->fb_flip->fb;
-
- for (i = 0; i < nplanes; i++) {
- struct drm_gem_cma_object *gem;
-
- dscr = slot->fb_flip->dscrs[i];
- gem = drm_fb_cma_get_gem_obj(fb, i);
- dscr->addr = gem->paddr + offsets[i];
- }
-
- fb_flip->ngems = nplanes;
- fb_flip->fb = fb;
-
- if (fb)
- drm_framebuffer_reference(fb);
-
- if (old_fb)
- drm_framebuffer_unreference(old_fb);
-}
-
-void atmel_hlcdc_layer_update_cfg(struct atmel_hlcdc_layer *layer, int cfg,
- u32 mask, u32 val)
-{
- struct atmel_hlcdc_layer_update *upd = &layer->update;
- struct atmel_hlcdc_layer_update_slot *slot;
-
- if (upd->next < 0 || upd->next > 1)
- return;
-
- if (cfg >= layer->desc->nconfigs)
- return;
-
- slot = &upd->slots[upd->next];
- slot->configs[cfg] &= ~mask;
- slot->configs[cfg] |= (val & mask);
- set_bit(cfg, slot->updated_configs);
-}
-
-void atmel_hlcdc_layer_update_commit(struct atmel_hlcdc_layer *layer)
-{
- struct atmel_hlcdc_layer_dma_channel *dma = &layer->dma;
- struct atmel_hlcdc_layer_update *upd = &layer->update;
- struct atmel_hlcdc_layer_update_slot *slot;
- unsigned long flags;
-
- if (upd->next < 0 || upd->next > 1)
- return;
-
- slot = &upd->slots[upd->next];
-
- spin_lock_irqsave(&layer->lock, flags);
-
- /*
- * Release pending update request and replace it by the new one.
- */
- if (upd->pending >= 0)
- atmel_hlcdc_layer_update_reset(layer, upd->pending);
-
- upd->pending = upd->next;
- upd->next = -1;
-
- if (!dma->queue)
- atmel_hlcdc_layer_update_apply(layer);
-
- spin_unlock_irqrestore(&layer->lock, flags);
-
-
- upd->next = -1;
-}
-
-static int atmel_hlcdc_layer_dma_init(struct drm_device *dev,
- struct atmel_hlcdc_layer *layer)
-{
- struct atmel_hlcdc_layer_dma_channel *dma = &layer->dma;
- dma_addr_t dma_addr;
- int i;
-
- dma->dscrs = dma_alloc_coherent(dev->dev,
- layer->max_planes * 4 *
- sizeof(*dma->dscrs),
- &dma_addr, GFP_KERNEL);
- if (!dma->dscrs)
- return -ENOMEM;
-
- for (i = 0; i < layer->max_planes * 4; i++) {
- struct atmel_hlcdc_dma_channel_dscr *dscr = &dma->dscrs[i];
-
- dscr->next = dma_addr + (i * sizeof(*dscr));
- }
-
- return 0;
-}
-
-static void atmel_hlcdc_layer_dma_cleanup(struct drm_device *dev,
- struct atmel_hlcdc_layer *layer)
-{
- struct atmel_hlcdc_layer_dma_channel *dma = &layer->dma;
- int i;
-
- for (i = 0; i < layer->max_planes * 4; i++) {
- struct atmel_hlcdc_dma_channel_dscr *dscr = &dma->dscrs[i];
-
- dscr->status = 0;
- }
-
- dma_free_coherent(dev->dev, layer->max_planes * 4 *
- sizeof(*dma->dscrs), dma->dscrs,
- dma->dscrs[0].next);
-}
-
-static int atmel_hlcdc_layer_update_init(struct drm_device *dev,
- struct atmel_hlcdc_layer *layer,
- const struct atmel_hlcdc_layer_desc *desc)
-{
- struct atmel_hlcdc_layer_update *upd = &layer->update;
- int updated_size;
- void *buffer;
- int i;
-
- updated_size = DIV_ROUND_UP(desc->nconfigs,
- BITS_PER_BYTE *
- sizeof(unsigned long));
-
- buffer = devm_kzalloc(dev->dev,
- ((desc->nconfigs * sizeof(u32)) +
- (updated_size * sizeof(unsigned long))) * 2,
- GFP_KERNEL);
- if (!buffer)
- return -ENOMEM;
-
- for (i = 0; i < 2; i++) {
- upd->slots[i].updated_configs = buffer;
- buffer += updated_size * sizeof(unsigned long);
- upd->slots[i].configs = buffer;
- buffer += desc->nconfigs * sizeof(u32);
- }
-
- upd->pending = -1;
- upd->next = -1;
-
- return 0;
-}
-
-int atmel_hlcdc_layer_init(struct drm_device *dev,
- struct atmel_hlcdc_layer *layer,
- const struct atmel_hlcdc_layer_desc *desc)
-{
- struct atmel_hlcdc_dc *dc = dev->dev_private;
- struct regmap *regmap = dc->hlcdc->regmap;
- unsigned int tmp;
- int ret;
- int i;
-
- layer->hlcdc = dc->hlcdc;
- layer->wq = dc->wq;
- layer->desc = desc;
-
- regmap_write(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_CHDR,
- ATMEL_HLCDC_LAYER_RST);
- for (i = 0; i < desc->formats->nformats; i++) {
- int nplanes = drm_format_num_planes(desc->formats->formats[i]);
-
- if (nplanes > layer->max_planes)
- layer->max_planes = nplanes;
- }
-
- spin_lock_init(&layer->lock);
- drm_flip_work_init(&layer->gc, desc->name,
- atmel_hlcdc_layer_fb_flip_release);
- ret = atmel_hlcdc_layer_dma_init(dev, layer);
- if (ret)
- return ret;
-
- ret = atmel_hlcdc_layer_update_init(dev, layer, desc);
- if (ret)
- return ret;
-
- /* Flush Status Register */
- regmap_write(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_IDR,
- 0xffffffff);
- regmap_read(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_ISR,
- &tmp);
-
- tmp = 0;
- for (i = 0; i < layer->max_planes; i++)
- tmp |= (ATMEL_HLCDC_LAYER_DMA_IRQ |
- ATMEL_HLCDC_LAYER_DSCR_IRQ |
- ATMEL_HLCDC_LAYER_ADD_IRQ |
- ATMEL_HLCDC_LAYER_DONE_IRQ |
- ATMEL_HLCDC_LAYER_OVR_IRQ) << (8 * i);
-
- regmap_write(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_IER, tmp);
-
- return 0;
-}
-
-void atmel_hlcdc_layer_cleanup(struct drm_device *dev,
- struct atmel_hlcdc_layer *layer)
-{
- const struct atmel_hlcdc_layer_desc *desc = layer->desc;
- struct regmap *regmap = layer->hlcdc->regmap;
-
- regmap_write(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_IDR,
- 0xffffffff);
- regmap_write(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_CHDR,
- ATMEL_HLCDC_LAYER_RST);
-
- atmel_hlcdc_layer_dma_cleanup(dev, layer);
- drm_flip_work_cleanup(&layer->gc);
-}
+++ /dev/null
-/*
- * Copyright (C) 2014 Free Electrons
- * Copyright (C) 2014 Atmel
- *
- * Author: Boris BREZILLON <boris.brezillon@free-electrons.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License version 2 as published by
- * the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- *
- * You should have received a copy of the GNU General Public License along with
- * this program. If not, see <http://www.gnu.org/licenses/>.
- */
-
-#ifndef DRM_ATMEL_HLCDC_LAYER_H
-#define DRM_ATMEL_HLCDC_LAYER_H
-
-#include <linux/mfd/atmel-hlcdc.h>
-
-#include <drm/drm_crtc.h>
-#include <drm/drm_flip_work.h>
-#include <drm/drmP.h>
-
-#define ATMEL_HLCDC_LAYER_CHER 0x0
-#define ATMEL_HLCDC_LAYER_CHDR 0x4
-#define ATMEL_HLCDC_LAYER_CHSR 0x8
-#define ATMEL_HLCDC_LAYER_DMA_CHAN BIT(0)
-#define ATMEL_HLCDC_LAYER_UPDATE BIT(1)
-#define ATMEL_HLCDC_LAYER_A2Q BIT(2)
-#define ATMEL_HLCDC_LAYER_RST BIT(8)
-
-#define ATMEL_HLCDC_LAYER_IER 0xc
-#define ATMEL_HLCDC_LAYER_IDR 0x10
-#define ATMEL_HLCDC_LAYER_IMR 0x14
-#define ATMEL_HLCDC_LAYER_ISR 0x18
-#define ATMEL_HLCDC_LAYER_DFETCH BIT(0)
-#define ATMEL_HLCDC_LAYER_LFETCH BIT(1)
-#define ATMEL_HLCDC_LAYER_DMA_IRQ BIT(2)
-#define ATMEL_HLCDC_LAYER_DSCR_IRQ BIT(3)
-#define ATMEL_HLCDC_LAYER_ADD_IRQ BIT(4)
-#define ATMEL_HLCDC_LAYER_DONE_IRQ BIT(5)
-#define ATMEL_HLCDC_LAYER_OVR_IRQ BIT(6)
-
-#define ATMEL_HLCDC_LAYER_PLANE_HEAD(n) (((n) * 0x10) + 0x1c)
-#define ATMEL_HLCDC_LAYER_PLANE_ADDR(n) (((n) * 0x10) + 0x20)
-#define ATMEL_HLCDC_LAYER_PLANE_CTRL(n) (((n) * 0x10) + 0x24)
-#define ATMEL_HLCDC_LAYER_PLANE_NEXT(n) (((n) * 0x10) + 0x28)
-#define ATMEL_HLCDC_LAYER_CFG(p, c) (((c) * 4) + ((p)->max_planes * 0x10) + 0x1c)
-
-#define ATMEL_HLCDC_LAYER_DMA_CFG_ID 0
-#define ATMEL_HLCDC_LAYER_DMA_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, ATMEL_HLCDC_LAYER_DMA_CFG_ID)
-#define ATMEL_HLCDC_LAYER_DMA_SIF BIT(0)
-#define ATMEL_HLCDC_LAYER_DMA_BLEN_MASK GENMASK(5, 4)
-#define ATMEL_HLCDC_LAYER_DMA_BLEN_SINGLE (0 << 4)
-#define ATMEL_HLCDC_LAYER_DMA_BLEN_INCR4 (1 << 4)
-#define ATMEL_HLCDC_LAYER_DMA_BLEN_INCR8 (2 << 4)
-#define ATMEL_HLCDC_LAYER_DMA_BLEN_INCR16 (3 << 4)
-#define ATMEL_HLCDC_LAYER_DMA_DLBO BIT(8)
-#define ATMEL_HLCDC_LAYER_DMA_ROTDIS BIT(12)
-#define ATMEL_HLCDC_LAYER_DMA_LOCKDIS BIT(13)
-
-#define ATMEL_HLCDC_LAYER_FORMAT_CFG_ID 1
-#define ATMEL_HLCDC_LAYER_FORMAT_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, ATMEL_HLCDC_LAYER_FORMAT_CFG_ID)
-#define ATMEL_HLCDC_LAYER_RGB (0 << 0)
-#define ATMEL_HLCDC_LAYER_CLUT (1 << 0)
-#define ATMEL_HLCDC_LAYER_YUV (2 << 0)
-#define ATMEL_HLCDC_RGB_MODE(m) (((m) & 0xf) << 4)
-#define ATMEL_HLCDC_CLUT_MODE(m) (((m) & 0x3) << 8)
-#define ATMEL_HLCDC_YUV_MODE(m) (((m) & 0xf) << 12)
-#define ATMEL_HLCDC_YUV422ROT BIT(16)
-#define ATMEL_HLCDC_YUV422SWP BIT(17)
-#define ATMEL_HLCDC_DSCALEOPT BIT(20)
-
-#define ATMEL_HLCDC_XRGB4444_MODE (ATMEL_HLCDC_LAYER_RGB | ATMEL_HLCDC_RGB_MODE(0))
-#define ATMEL_HLCDC_ARGB4444_MODE (ATMEL_HLCDC_LAYER_RGB | ATMEL_HLCDC_RGB_MODE(1))
-#define ATMEL_HLCDC_RGBA4444_MODE (ATMEL_HLCDC_LAYER_RGB | ATMEL_HLCDC_RGB_MODE(2))
-#define ATMEL_HLCDC_RGB565_MODE (ATMEL_HLCDC_LAYER_RGB | ATMEL_HLCDC_RGB_MODE(3))
-#define ATMEL_HLCDC_ARGB1555_MODE (ATMEL_HLCDC_LAYER_RGB | ATMEL_HLCDC_RGB_MODE(4))
-#define ATMEL_HLCDC_XRGB8888_MODE (ATMEL_HLCDC_LAYER_RGB | ATMEL_HLCDC_RGB_MODE(9))
-#define ATMEL_HLCDC_RGB888_MODE (ATMEL_HLCDC_LAYER_RGB | ATMEL_HLCDC_RGB_MODE(10))
-#define ATMEL_HLCDC_ARGB8888_MODE (ATMEL_HLCDC_LAYER_RGB | ATMEL_HLCDC_RGB_MODE(12))
-#define ATMEL_HLCDC_RGBA8888_MODE (ATMEL_HLCDC_LAYER_RGB | ATMEL_HLCDC_RGB_MODE(13))
-
-#define ATMEL_HLCDC_AYUV_MODE (ATMEL_HLCDC_LAYER_YUV | ATMEL_HLCDC_YUV_MODE(0))
-#define ATMEL_HLCDC_YUYV_MODE (ATMEL_HLCDC_LAYER_YUV | ATMEL_HLCDC_YUV_MODE(1))
-#define ATMEL_HLCDC_UYVY_MODE (ATMEL_HLCDC_LAYER_YUV | ATMEL_HLCDC_YUV_MODE(2))
-#define ATMEL_HLCDC_YVYU_MODE (ATMEL_HLCDC_LAYER_YUV | ATMEL_HLCDC_YUV_MODE(3))
-#define ATMEL_HLCDC_VYUY_MODE (ATMEL_HLCDC_LAYER_YUV | ATMEL_HLCDC_YUV_MODE(4))
-#define ATMEL_HLCDC_NV61_MODE (ATMEL_HLCDC_LAYER_YUV | ATMEL_HLCDC_YUV_MODE(5))
-#define ATMEL_HLCDC_YUV422_MODE (ATMEL_HLCDC_LAYER_YUV | ATMEL_HLCDC_YUV_MODE(6))
-#define ATMEL_HLCDC_NV21_MODE (ATMEL_HLCDC_LAYER_YUV | ATMEL_HLCDC_YUV_MODE(7))
-#define ATMEL_HLCDC_YUV420_MODE (ATMEL_HLCDC_LAYER_YUV | ATMEL_HLCDC_YUV_MODE(8))
-
-#define ATMEL_HLCDC_LAYER_POS_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.pos)
-#define ATMEL_HLCDC_LAYER_SIZE_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.size)
-#define ATMEL_HLCDC_LAYER_MEMSIZE_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.memsize)
-#define ATMEL_HLCDC_LAYER_XSTRIDE_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.xstride)
-#define ATMEL_HLCDC_LAYER_PSTRIDE_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.pstride)
-#define ATMEL_HLCDC_LAYER_DFLTCOLOR_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.default_color)
-#define ATMEL_HLCDC_LAYER_CRKEY_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.chroma_key)
-#define ATMEL_HLCDC_LAYER_CRKEY_MASK_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.chroma_key_mask)
-
-#define ATMEL_HLCDC_LAYER_GENERAL_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.general_config)
-#define ATMEL_HLCDC_LAYER_CRKEY BIT(0)
-#define ATMEL_HLCDC_LAYER_INV BIT(1)
-#define ATMEL_HLCDC_LAYER_ITER2BL BIT(2)
-#define ATMEL_HLCDC_LAYER_ITER BIT(3)
-#define ATMEL_HLCDC_LAYER_REVALPHA BIT(4)
-#define ATMEL_HLCDC_LAYER_GAEN BIT(5)
-#define ATMEL_HLCDC_LAYER_LAEN BIT(6)
-#define ATMEL_HLCDC_LAYER_OVR BIT(7)
-#define ATMEL_HLCDC_LAYER_DMA BIT(8)
-#define ATMEL_HLCDC_LAYER_REP BIT(9)
-#define ATMEL_HLCDC_LAYER_DSTKEY BIT(10)
-#define ATMEL_HLCDC_LAYER_DISCEN BIT(11)
-#define ATMEL_HLCDC_LAYER_GA_SHIFT 16
-#define ATMEL_HLCDC_LAYER_GA_MASK GENMASK(23, ATMEL_HLCDC_LAYER_GA_SHIFT)
-#define ATMEL_HLCDC_LAYER_GA(x) ((x) << ATMEL_HLCDC_LAYER_GA_SHIFT)
-
-#define ATMEL_HLCDC_LAYER_CSC_CFG(p, o) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.csc + o)
-
-#define ATMEL_HLCDC_LAYER_DISC_POS_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.disc_pos)
-
-#define ATMEL_HLCDC_LAYER_DISC_SIZE_CFG(p) ATMEL_HLCDC_LAYER_CFG(p, (p)->desc->layout.disc_size)
-
-#define ATMEL_HLCDC_MAX_PLANES 3
-
-#define ATMEL_HLCDC_DMA_CHANNEL_DSCR_RESERVED BIT(0)
-#define ATMEL_HLCDC_DMA_CHANNEL_DSCR_LOADED BIT(1)
-#define ATMEL_HLCDC_DMA_CHANNEL_DSCR_DONE BIT(2)
-#define ATMEL_HLCDC_DMA_CHANNEL_DSCR_OVERRUN BIT(3)
-
-/**
- * Atmel HLCDC Layer registers layout structure
- *
- * Each HLCDC layer has its own register organization and a given register
- * can be placed differently on 2 different layers depending on its
- * capabilities.
- * This structure stores common registers layout for a given layer and is
- * used by HLCDC layer code to choose the appropriate register to write to
- * or to read from.
- *
- * For all fields, a value of zero means "unsupported".
- *
- * See Atmel's datasheet for a detailled description of these registers.
- *
- * @xstride: xstride registers
- * @pstride: pstride registers
- * @pos: position register
- * @size: displayed size register
- * @memsize: memory size register
- * @default_color: default color register
- * @chroma_key: chroma key register
- * @chroma_key_mask: chroma key mask register
- * @general_config: general layer config register
- * @disc_pos: discard area position register
- * @disc_size: discard area size register
- * @csc: color space conversion register
- */
-struct atmel_hlcdc_layer_cfg_layout {
- int xstride[ATMEL_HLCDC_MAX_PLANES];
- int pstride[ATMEL_HLCDC_MAX_PLANES];
- int pos;
- int size;
- int memsize;
- int default_color;
- int chroma_key;
- int chroma_key_mask;
- int general_config;
- int disc_pos;
- int disc_size;
- int csc;
-};
-
-/**
- * Atmel HLCDC framebuffer flip structure
- *
- * This structure is allocated when someone asked for a layer update (most
- * likely a DRM plane update, either primary, overlay or cursor plane) and
- * released when the layer do not need to reference the framebuffer object
- * anymore (i.e. the layer was disabled or updated).
- *
- * @dscrs: DMA descriptors
- * @fb: the referenced framebuffer object
- * @ngems: number of GEM objects referenced by the fb element
- * @status: fb flip operation status
- */
-struct atmel_hlcdc_layer_fb_flip {
- struct atmel_hlcdc_dma_channel_dscr *dscrs[ATMEL_HLCDC_MAX_PLANES];
- struct drm_flip_task *task;
- struct drm_framebuffer *fb;
- int ngems;
- u32 status;
-};
-
-/**
- * Atmel HLCDC DMA descriptor structure
- *
- * This structure is used by the HLCDC DMA engine to schedule a DMA transfer.
- *
- * The structure fields must remain in this specific order, because they're
- * used by the HLCDC DMA engine, which expect them in this order.
- * HLCDC DMA descriptors must be aligned on 64 bits.
- *
- * @addr: buffer DMA address
- * @ctrl: DMA transfer options
- * @next: next DMA descriptor to fetch
- * @gem_flip: the attached gem_flip operation
- */
-struct atmel_hlcdc_dma_channel_dscr {
- dma_addr_t addr;
- u32 ctrl;
- dma_addr_t next;
- u32 status;
-} __aligned(sizeof(u64));
-
-/**
- * Atmel HLCDC layer types
- */
-enum atmel_hlcdc_layer_type {
- ATMEL_HLCDC_BASE_LAYER,
- ATMEL_HLCDC_OVERLAY_LAYER,
- ATMEL_HLCDC_CURSOR_LAYER,
- ATMEL_HLCDC_PP_LAYER,
-};
-
-/**
- * Atmel HLCDC Supported formats structure
- *
- * This structure list all the formats supported by a given layer.
- *
- * @nformats: number of supported formats
- * @formats: supported formats
- */
-struct atmel_hlcdc_formats {
- int nformats;
- uint32_t *formats;
-};
-
-/**
- * Atmel HLCDC Layer description structure
- *
- * This structure describe the capabilities provided by a given layer.
- *
- * @name: layer name
- * @type: layer type
- * @id: layer id
- * @regs_offset: offset of the layer registers from the HLCDC registers base
- * @nconfigs: number of config registers provided by this layer
- * @formats: supported formats
- * @layout: config registers layout
- * @max_width: maximum width supported by this layer (0 means unlimited)
- * @max_height: maximum height supported by this layer (0 means unlimited)
- */
-struct atmel_hlcdc_layer_desc {
- const char *name;
- enum atmel_hlcdc_layer_type type;
- int id;
- int regs_offset;
- int nconfigs;
- struct atmel_hlcdc_formats *formats;
- struct atmel_hlcdc_layer_cfg_layout layout;
- int max_width;
- int max_height;
-};
-
-/**
- * Atmel HLCDC Layer Update Slot structure
- *
- * This structure stores layer update requests to be applied on next frame.
- * This is the base structure behind the atomic layer update infrastructure.
- *
- * Atomic layer update provides a way to update all layer's parameters
- * simultaneously. This is needed to avoid incompatible sequential updates
- * like this one:
- * 1) update layer format from RGB888 (1 plane/buffer) to YUV422
- * (2 planes/buffers)
- * 2) the format update is applied but the DMA channel for the second
- * plane/buffer is not enabled
- * 3) enable the DMA channel for the second plane
- *
- * @fb_flip: fb_flip object
- * @updated_configs: bitmask used to record modified configs
- * @configs: new config values
- */
-struct atmel_hlcdc_layer_update_slot {
- struct atmel_hlcdc_layer_fb_flip *fb_flip;
- unsigned long *updated_configs;
- u32 *configs;
-};
-
-/**
- * Atmel HLCDC Layer Update structure
- *
- * This structure provides a way to queue layer update requests.
- *
- * At a given time there is at most:
- * - one pending update request, which means the update request has been
- * committed (or validated) and is waiting for the DMA channel(s) to be
- * available
- * - one request being prepared, which means someone started a layer update
- * but has not committed it yet. There cannot be more than one started
- * request, because the update lock is taken when starting a layer update
- * and release when committing or rolling back the request.
- *
- * @slots: update slots. One is used for pending request and the other one
- * for started update request
- * @pending: the pending slot index or -1 if no request is pending
- * @next: the started update slot index or -1 no update has been started
- */
-struct atmel_hlcdc_layer_update {
- struct atmel_hlcdc_layer_update_slot slots[2];
- int pending;
- int next;
-};
-
-enum atmel_hlcdc_layer_dma_channel_status {
- ATMEL_HLCDC_LAYER_DISABLED,
- ATMEL_HLCDC_LAYER_ENABLED,
- ATMEL_HLCDC_LAYER_DISABLING,
-};
-
-/**
- * Atmel HLCDC Layer DMA channel structure
- *
- * This structure stores information on the DMA channel associated to a
- * given layer.
- *
- * @status: DMA channel status
- * @cur: current framebuffer
- * @queue: next framebuffer
- * @dscrs: allocated DMA descriptors
- */
-struct atmel_hlcdc_layer_dma_channel {
- enum atmel_hlcdc_layer_dma_channel_status status;
- struct atmel_hlcdc_layer_fb_flip *cur;
- struct atmel_hlcdc_layer_fb_flip *queue;
- struct atmel_hlcdc_dma_channel_dscr *dscrs;
-};
-
-/**
- * Atmel HLCDC Layer structure
- *
- * This structure stores information on the layer instance.
- *
- * @desc: layer description
- * @max_planes: maximum planes/buffers that can be associated with this layer.
- * This depends on the supported formats.
- * @hlcdc: pointer to the atmel_hlcdc structure provided by the MFD device
- * @dma: dma channel
- * @gc: fb flip garbage collector
- * @update: update handler
- * @lock: layer lock
- */
-struct atmel_hlcdc_layer {
- const struct atmel_hlcdc_layer_desc *desc;
- int max_planes;
- struct atmel_hlcdc *hlcdc;
- struct workqueue_struct *wq;
- struct drm_flip_work gc;
- struct atmel_hlcdc_layer_dma_channel dma;
- struct atmel_hlcdc_layer_update update;
- spinlock_t lock;
-};
-
-void atmel_hlcdc_layer_irq(struct atmel_hlcdc_layer *layer);
-
-int atmel_hlcdc_layer_init(struct drm_device *dev,
- struct atmel_hlcdc_layer *layer,
- const struct atmel_hlcdc_layer_desc *desc);
-
-void atmel_hlcdc_layer_cleanup(struct drm_device *dev,
- struct atmel_hlcdc_layer *layer);
-
-void atmel_hlcdc_layer_disable(struct atmel_hlcdc_layer *layer);
-
-int atmel_hlcdc_layer_update_start(struct atmel_hlcdc_layer *layer);
-
-void atmel_hlcdc_layer_update_cfg(struct atmel_hlcdc_layer *layer, int cfg,
- u32 mask, u32 val);
-
-void atmel_hlcdc_layer_update_set_fb(struct atmel_hlcdc_layer *layer,
- struct drm_framebuffer *fb,
- unsigned int *offsets);
-
-void atmel_hlcdc_layer_update_set_finished(struct atmel_hlcdc_layer *layer,
- void (*finished)(void *data),
- void *finished_data);
-
-void atmel_hlcdc_layer_update_rollback(struct atmel_hlcdc_layer *layer);
-
-void atmel_hlcdc_layer_update_commit(struct atmel_hlcdc_layer *layer);
-
-#endif /* DRM_ATMEL_HLCDC_LAYER_H */
* @src_w: buffer width
* @src_h: buffer height
* @alpha: alpha blending of the plane
+ * @disc_x: x discard position
+ * @disc_y: y discard position
+ * @disc_w: discard width
+ * @disc_h: discard height
* @bpp: bytes per pixel deduced from pixel_format
* @offsets: offsets to apply to the GEM buffers
* @xstride: value to add to the pixel pointer between each line
* @pstride: value to add to the pixel pointer between each pixel
* @nplanes: number of planes (deduced from pixel_format)
- * @prepared: plane update has been prepared
+ * @dscrs: DMA descriptors
*/
struct atmel_hlcdc_plane_state {
struct drm_plane_state base;
u8 alpha;
- bool disc_updated;
-
int disc_x;
int disc_y;
int disc_w;
int ahb_id;
/* These fields are private and should not be touched */
- int bpp[ATMEL_HLCDC_MAX_PLANES];
- unsigned int offsets[ATMEL_HLCDC_MAX_PLANES];
- int xstride[ATMEL_HLCDC_MAX_PLANES];
- int pstride[ATMEL_HLCDC_MAX_PLANES];
+ int bpp[ATMEL_HLCDC_LAYER_MAX_PLANES];
+ unsigned int offsets[ATMEL_HLCDC_LAYER_MAX_PLANES];
+ int xstride[ATMEL_HLCDC_LAYER_MAX_PLANES];
+ int pstride[ATMEL_HLCDC_LAYER_MAX_PLANES];
int nplanes;
- bool prepared;
+
+ /* DMA descriptors. */
+ struct atmel_hlcdc_dma_channel_dscr *dscrs[ATMEL_HLCDC_LAYER_MAX_PLANES];
};
static inline struct atmel_hlcdc_plane_state *
0x00205907,
};
+#define ATMEL_HLCDC_XPHIDEF 4
+#define ATMEL_HLCDC_YPHIDEF 4
+
+static u32 atmel_hlcdc_plane_phiscaler_get_factor(u32 srcsize,
+ u32 dstsize,
+ u32 phidef)
+{
+ u32 factor, max_memsize;
+
+ factor = (256 * ((8 * (srcsize - 1)) - phidef)) / (dstsize - 1);
+ max_memsize = ((factor * (dstsize - 1)) + (256 * phidef)) / 2048;
+
+ if (max_memsize > srcsize - 1)
+ factor--;
+
+ return factor;
+}
+
static void
-atmel_hlcdc_plane_update_pos_and_size(struct atmel_hlcdc_plane *plane,
- struct atmel_hlcdc_plane_state *state)
+atmel_hlcdc_plane_scaler_set_phicoeff(struct atmel_hlcdc_plane *plane,
+ const u32 *coeff_tab, int size,
+ unsigned int cfg_offs)
{
- const struct atmel_hlcdc_layer_cfg_layout *layout =
- &plane->layer.desc->layout;
-
- if (layout->size)
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- layout->size,
- 0xffffffff,
- (state->crtc_w - 1) |
- ((state->crtc_h - 1) << 16));
-
- if (layout->memsize)
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- layout->memsize,
- 0xffffffff,
- (state->src_w - 1) |
- ((state->src_h - 1) << 16));
-
- if (layout->pos)
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- layout->pos,
- 0xffffffff,
- state->crtc_x |
- (state->crtc_y << 16));
-
- /* TODO: rework the rescaling part */
- if (state->crtc_w != state->src_w || state->crtc_h != state->src_h) {
- u32 factor_reg = 0;
-
- if (state->crtc_w != state->src_w) {
- int i;
- u32 factor;
- u32 *coeff_tab = heo_upscaling_xcoef;
- u32 max_memsize;
-
- if (state->crtc_w < state->src_w)
- coeff_tab = heo_downscaling_xcoef;
- for (i = 0; i < ARRAY_SIZE(heo_upscaling_xcoef); i++)
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- 17 + i,
- 0xffffffff,
- coeff_tab[i]);
- factor = ((8 * 256 * state->src_w) - (256 * 4)) /
- state->crtc_w;
- factor++;
- max_memsize = ((factor * state->crtc_w) + (256 * 4)) /
- 2048;
- if (max_memsize > state->src_w)
- factor--;
- factor_reg |= factor | 0x80000000;
- }
+ int i;
- if (state->crtc_h != state->src_h) {
- int i;
- u32 factor;
- u32 *coeff_tab = heo_upscaling_ycoef;
- u32 max_memsize;
-
- if (state->crtc_h < state->src_h)
- coeff_tab = heo_downscaling_ycoef;
- for (i = 0; i < ARRAY_SIZE(heo_upscaling_ycoef); i++)
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- 33 + i,
- 0xffffffff,
- coeff_tab[i]);
- factor = ((8 * 256 * state->src_h) - (256 * 4)) /
- state->crtc_h;
- factor++;
- max_memsize = ((factor * state->crtc_h) + (256 * 4)) /
- 2048;
- if (max_memsize > state->src_h)
- factor--;
- factor_reg |= (factor << 16) | 0x80000000;
- }
+ for (i = 0; i < size; i++)
+ atmel_hlcdc_layer_write_cfg(&plane->layer, cfg_offs + i,
+ coeff_tab[i]);
+}
+
+void atmel_hlcdc_plane_setup_scaler(struct atmel_hlcdc_plane *plane,
+ struct atmel_hlcdc_plane_state *state)
+{
+ const struct atmel_hlcdc_layer_desc *desc = plane->layer.desc;
+ u32 xfactor, yfactor;
+
+ if (!desc->layout.scaler_config)
+ return;
- atmel_hlcdc_layer_update_cfg(&plane->layer, 13, 0xffffffff,
- factor_reg);
+ if (state->crtc_w == state->src_w && state->crtc_h == state->src_h) {
+ atmel_hlcdc_layer_write_cfg(&plane->layer,
+ desc->layout.scaler_config, 0);
+ return;
+ }
+
+ if (desc->layout.phicoeffs.x) {
+ xfactor = atmel_hlcdc_plane_phiscaler_get_factor(state->src_w,
+ state->crtc_w,
+ ATMEL_HLCDC_XPHIDEF);
+
+ yfactor = atmel_hlcdc_plane_phiscaler_get_factor(state->src_h,
+ state->crtc_h,
+ ATMEL_HLCDC_YPHIDEF);
+
+ atmel_hlcdc_plane_scaler_set_phicoeff(plane,
+ state->crtc_w < state->src_w ?
+ heo_downscaling_xcoef :
+ heo_upscaling_xcoef,
+ ARRAY_SIZE(heo_upscaling_xcoef),
+ desc->layout.phicoeffs.x);
+
+ atmel_hlcdc_plane_scaler_set_phicoeff(plane,
+ state->crtc_h < state->src_h ?
+ heo_downscaling_ycoef :
+ heo_upscaling_ycoef,
+ ARRAY_SIZE(heo_upscaling_ycoef),
+ desc->layout.phicoeffs.y);
} else {
- atmel_hlcdc_layer_update_cfg(&plane->layer, 13, 0xffffffff, 0);
+ xfactor = (1024 * state->src_w) / state->crtc_w;
+ yfactor = (1024 * state->src_h) / state->crtc_h;
}
+
+ atmel_hlcdc_layer_write_cfg(&plane->layer, desc->layout.scaler_config,
+ ATMEL_HLCDC_LAYER_SCALER_ENABLE |
+ ATMEL_HLCDC_LAYER_SCALER_FACTORS(xfactor,
+ yfactor));
+}
+
+static void
+atmel_hlcdc_plane_update_pos_and_size(struct atmel_hlcdc_plane *plane,
+ struct atmel_hlcdc_plane_state *state)
+{
+ const struct atmel_hlcdc_layer_desc *desc = plane->layer.desc;
+
+ if (desc->layout.size)
+ atmel_hlcdc_layer_write_cfg(&plane->layer, desc->layout.size,
+ ATMEL_HLCDC_LAYER_SIZE(state->crtc_w,
+ state->crtc_h));
+
+ if (desc->layout.memsize)
+ atmel_hlcdc_layer_write_cfg(&plane->layer,
+ desc->layout.memsize,
+ ATMEL_HLCDC_LAYER_SIZE(state->src_w,
+ state->src_h));
+
+ if (desc->layout.pos)
+ atmel_hlcdc_layer_write_cfg(&plane->layer, desc->layout.pos,
+ ATMEL_HLCDC_LAYER_POS(state->crtc_x,
+ state->crtc_y));
+
+ atmel_hlcdc_plane_setup_scaler(plane, state);
}
static void
atmel_hlcdc_plane_update_general_settings(struct atmel_hlcdc_plane *plane,
struct atmel_hlcdc_plane_state *state)
{
- const struct atmel_hlcdc_layer_cfg_layout *layout =
- &plane->layer.desc->layout;
- unsigned int cfg = ATMEL_HLCDC_LAYER_DMA;
+ unsigned int cfg = ATMEL_HLCDC_LAYER_DMA_BLEN_INCR16 | state->ahb_id;
+ const struct atmel_hlcdc_layer_desc *desc = plane->layer.desc;
+ u32 format = state->base.fb->format->format;
+
+ /*
+ * Rotation optimization is not working on RGB888 (rotation is still
+ * working but without any optimization).
+ */
+ if (format == DRM_FORMAT_RGB888)
+ cfg |= ATMEL_HLCDC_LAYER_DMA_ROTDIS;
+
+ atmel_hlcdc_layer_write_cfg(&plane->layer, ATMEL_HLCDC_LAYER_DMA_CFG,
+ cfg);
+
+ cfg = ATMEL_HLCDC_LAYER_DMA;
if (plane->base.type != DRM_PLANE_TYPE_PRIMARY) {
cfg |= ATMEL_HLCDC_LAYER_OVR | ATMEL_HLCDC_LAYER_ITER2BL |
ATMEL_HLCDC_LAYER_ITER;
- if (atmel_hlcdc_format_embeds_alpha(state->base.fb->format->format))
+ if (atmel_hlcdc_format_embeds_alpha(format))
cfg |= ATMEL_HLCDC_LAYER_LAEN;
else
cfg |= ATMEL_HLCDC_LAYER_GAEN |
ATMEL_HLCDC_LAYER_GA(state->alpha);
}
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- ATMEL_HLCDC_LAYER_DMA_CFG_ID,
- ATMEL_HLCDC_LAYER_DMA_BLEN_MASK |
- ATMEL_HLCDC_LAYER_DMA_SIF,
- ATMEL_HLCDC_LAYER_DMA_BLEN_INCR16 |
- state->ahb_id);
-
- atmel_hlcdc_layer_update_cfg(&plane->layer, layout->general_config,
- ATMEL_HLCDC_LAYER_ITER2BL |
- ATMEL_HLCDC_LAYER_ITER |
- ATMEL_HLCDC_LAYER_GAEN |
- ATMEL_HLCDC_LAYER_GA_MASK |
- ATMEL_HLCDC_LAYER_LAEN |
- ATMEL_HLCDC_LAYER_OVR |
- ATMEL_HLCDC_LAYER_DMA, cfg);
+ if (state->disc_h && state->disc_w)
+ cfg |= ATMEL_HLCDC_LAYER_DISCEN;
+
+ atmel_hlcdc_layer_write_cfg(&plane->layer, desc->layout.general_config,
+ cfg);
}
static void atmel_hlcdc_plane_update_format(struct atmel_hlcdc_plane *plane,
drm_rotation_90_or_270(state->base.rotation))
cfg |= ATMEL_HLCDC_YUV422ROT;
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- ATMEL_HLCDC_LAYER_FORMAT_CFG_ID,
- 0xffffffff,
- cfg);
-
- /*
- * Rotation optimization is not working on RGB888 (rotation is still
- * working but without any optimization).
- */
- if (state->base.fb->format->format == DRM_FORMAT_RGB888)
- cfg = ATMEL_HLCDC_LAYER_DMA_ROTDIS;
- else
- cfg = 0;
-
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- ATMEL_HLCDC_LAYER_DMA_CFG_ID,
- ATMEL_HLCDC_LAYER_DMA_ROTDIS, cfg);
+ atmel_hlcdc_layer_write_cfg(&plane->layer,
+ ATMEL_HLCDC_LAYER_FORMAT_CFG, cfg);
}
static void atmel_hlcdc_plane_update_buffers(struct atmel_hlcdc_plane *plane,
struct atmel_hlcdc_plane_state *state)
{
- struct atmel_hlcdc_layer *layer = &plane->layer;
- const struct atmel_hlcdc_layer_cfg_layout *layout =
- &layer->desc->layout;
+ const struct atmel_hlcdc_layer_desc *desc = plane->layer.desc;
+ struct drm_framebuffer *fb = state->base.fb;
+ u32 sr;
int i;
- atmel_hlcdc_layer_update_set_fb(&plane->layer, state->base.fb,
- state->offsets);
+ sr = atmel_hlcdc_layer_read_reg(&plane->layer, ATMEL_HLCDC_LAYER_CHSR);
for (i = 0; i < state->nplanes; i++) {
- if (layout->xstride[i]) {
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- layout->xstride[i],
- 0xffffffff,
- state->xstride[i]);
+ struct drm_gem_cma_object *gem = drm_fb_cma_get_gem_obj(fb, i);
+
+ state->dscrs[i]->addr = gem->paddr + state->offsets[i];
+
+ atmel_hlcdc_layer_write_reg(&plane->layer,
+ ATMEL_HLCDC_LAYER_PLANE_HEAD(i),
+ state->dscrs[i]->self);
+
+ if (!(sr & ATMEL_HLCDC_LAYER_EN)) {
+ atmel_hlcdc_layer_write_reg(&plane->layer,
+ ATMEL_HLCDC_LAYER_PLANE_ADDR(i),
+ state->dscrs[i]->addr);
+ atmel_hlcdc_layer_write_reg(&plane->layer,
+ ATMEL_HLCDC_LAYER_PLANE_CTRL(i),
+ state->dscrs[i]->ctrl);
+ atmel_hlcdc_layer_write_reg(&plane->layer,
+ ATMEL_HLCDC_LAYER_PLANE_NEXT(i),
+ state->dscrs[i]->self);
}
- if (layout->pstride[i]) {
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- layout->pstride[i],
- 0xffffffff,
- state->pstride[i]);
- }
+ if (desc->layout.xstride[i])
+ atmel_hlcdc_layer_write_cfg(&plane->layer,
+ desc->layout.xstride[i],
+ state->xstride[i]);
+
+ if (desc->layout.pstride[i])
+ atmel_hlcdc_layer_write_cfg(&plane->layer,
+ desc->layout.pstride[i],
+ state->pstride[i]);
}
}
disc_w = ovl_state->crtc_w;
}
- if (disc_x == primary_state->disc_x &&
- disc_y == primary_state->disc_y &&
- disc_w == primary_state->disc_w &&
- disc_h == primary_state->disc_h)
- return 0;
-
-
primary_state->disc_x = disc_x;
primary_state->disc_y = disc_y;
primary_state->disc_w = disc_w;
primary_state->disc_h = disc_h;
- primary_state->disc_updated = true;
return 0;
}
atmel_hlcdc_plane_update_disc_area(struct atmel_hlcdc_plane *plane,
struct atmel_hlcdc_plane_state *state)
{
- const struct atmel_hlcdc_layer_cfg_layout *layout =
- &plane->layer.desc->layout;
- int disc_surface = 0;
-
- if (!state->disc_updated)
- return;
-
- disc_surface = state->disc_h * state->disc_w;
-
- atmel_hlcdc_layer_update_cfg(&plane->layer, layout->general_config,
- ATMEL_HLCDC_LAYER_DISCEN,
- disc_surface ? ATMEL_HLCDC_LAYER_DISCEN : 0);
+ const struct atmel_hlcdc_layer_cfg_layout *layout;
- if (!disc_surface)
+ layout = &plane->layer.desc->layout;
+ if (!layout->disc_pos || !layout->disc_size)
return;
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- layout->disc_pos,
- 0xffffffff,
- state->disc_x | (state->disc_y << 16));
+ atmel_hlcdc_layer_write_cfg(&plane->layer, layout->disc_pos,
+ ATMEL_HLCDC_LAYER_DISC_POS(state->disc_x,
+ state->disc_y));
- atmel_hlcdc_layer_update_cfg(&plane->layer,
- layout->disc_size,
- 0xffffffff,
- (state->disc_w - 1) |
- ((state->disc_h - 1) << 16));
+ atmel_hlcdc_layer_write_cfg(&plane->layer, layout->disc_size,
+ ATMEL_HLCDC_LAYER_DISC_SIZE(state->disc_w,
+ state->disc_h));
}
static int atmel_hlcdc_plane_atomic_check(struct drm_plane *p,
struct atmel_hlcdc_plane *plane = drm_plane_to_atmel_hlcdc_plane(p);
struct atmel_hlcdc_plane_state *state =
drm_plane_state_to_atmel_hlcdc_plane_state(s);
- const struct atmel_hlcdc_layer_cfg_layout *layout =
- &plane->layer.desc->layout;
+ const struct atmel_hlcdc_layer_desc *desc = plane->layer.desc;
struct drm_framebuffer *fb = state->base.fb;
const struct drm_display_mode *mode;
struct drm_crtc_state *crtc_state;
state->src_h >>= 16;
state->nplanes = fb->format->num_planes;
- if (state->nplanes > ATMEL_HLCDC_MAX_PLANES)
+ if (state->nplanes > ATMEL_HLCDC_LAYER_MAX_PLANES)
return -EINVAL;
/*
state->crtc_w = patched_crtc_w;
state->crtc_h = patched_crtc_h;
- if (!layout->size &&
+ if (!desc->layout.size &&
(mode->hdisplay != state->crtc_w ||
mode->vdisplay != state->crtc_h))
return -EINVAL;
- if (plane->layer.desc->max_height &&
- state->crtc_h > plane->layer.desc->max_height)
+ if (desc->max_height && state->crtc_h > desc->max_height)
return -EINVAL;
- if (plane->layer.desc->max_width &&
- state->crtc_w > plane->layer.desc->max_width)
+ if (desc->max_width && state->crtc_w > desc->max_width)
return -EINVAL;
if ((state->crtc_h != state->src_h || state->crtc_w != state->src_w) &&
- (!layout->memsize ||
+ (!desc->layout.memsize ||
atmel_hlcdc_format_embeds_alpha(state->base.fb->format->format)))
return -EINVAL;
return 0;
}
-static int atmel_hlcdc_plane_prepare_fb(struct drm_plane *p,
- struct drm_plane_state *new_state)
-{
- /*
- * FIXME: we should avoid this const -> non-const cast but it's
- * currently the only solution we have to modify the ->prepared
- * state and rollback the update request.
- * Ideally, we should rework the code to attach all the resources
- * to atmel_hlcdc_plane_state (including the DMA desc allocation),
- * but this require a complete rework of the atmel_hlcdc_layer
- * code.
- */
- struct drm_plane_state *s = (struct drm_plane_state *)new_state;
- struct atmel_hlcdc_plane *plane = drm_plane_to_atmel_hlcdc_plane(p);
- struct atmel_hlcdc_plane_state *state =
- drm_plane_state_to_atmel_hlcdc_plane_state(s);
- int ret;
-
- ret = atmel_hlcdc_layer_update_start(&plane->layer);
- if (!ret)
- state->prepared = true;
-
- return ret;
-}
-
-static void atmel_hlcdc_plane_cleanup_fb(struct drm_plane *p,
- struct drm_plane_state *old_state)
-{
- /*
- * FIXME: we should avoid this const -> non-const cast but it's
- * currently the only solution we have to modify the ->prepared
- * state and rollback the update request.
- * Ideally, we should rework the code to attach all the resources
- * to atmel_hlcdc_plane_state (including the DMA desc allocation),
- * but this require a complete rework of the atmel_hlcdc_layer
- * code.
- */
- struct drm_plane_state *s = (struct drm_plane_state *)old_state;
- struct atmel_hlcdc_plane *plane = drm_plane_to_atmel_hlcdc_plane(p);
- struct atmel_hlcdc_plane_state *state =
- drm_plane_state_to_atmel_hlcdc_plane_state(s);
-
- /*
- * The Request has already been applied or cancelled, nothing to do
- * here.
- */
- if (!state->prepared)
- return;
-
- atmel_hlcdc_layer_update_rollback(&plane->layer);
- state->prepared = false;
-}
-
static void atmel_hlcdc_plane_atomic_update(struct drm_plane *p,
struct drm_plane_state *old_s)
{
struct atmel_hlcdc_plane *plane = drm_plane_to_atmel_hlcdc_plane(p);
struct atmel_hlcdc_plane_state *state =
drm_plane_state_to_atmel_hlcdc_plane_state(p->state);
+ u32 sr;
if (!p->state->crtc || !p->state->fb)
return;
atmel_hlcdc_plane_update_buffers(plane, state);
atmel_hlcdc_plane_update_disc_area(plane, state);
- atmel_hlcdc_layer_update_commit(&plane->layer);
+ /* Enable the overrun interrupts. */
+ atmel_hlcdc_layer_write_reg(&plane->layer, ATMEL_HLCDC_LAYER_IER,
+ ATMEL_HLCDC_LAYER_OVR_IRQ(0) |
+ ATMEL_HLCDC_LAYER_OVR_IRQ(1) |
+ ATMEL_HLCDC_LAYER_OVR_IRQ(2));
+
+ /* Apply the new config at the next SOF event. */
+ sr = atmel_hlcdc_layer_read_reg(&plane->layer, ATMEL_HLCDC_LAYER_CHSR);
+ atmel_hlcdc_layer_write_reg(&plane->layer, ATMEL_HLCDC_LAYER_CHER,
+ ATMEL_HLCDC_LAYER_UPDATE |
+ (sr & ATMEL_HLCDC_LAYER_EN ?
+ ATMEL_HLCDC_LAYER_A2Q : ATMEL_HLCDC_LAYER_EN));
}
static void atmel_hlcdc_plane_atomic_disable(struct drm_plane *p,
{
struct atmel_hlcdc_plane *plane = drm_plane_to_atmel_hlcdc_plane(p);
- atmel_hlcdc_layer_disable(&plane->layer);
+ /* Disable interrupts */
+ atmel_hlcdc_layer_write_reg(&plane->layer, ATMEL_HLCDC_LAYER_IDR,
+ 0xffffffff);
+
+ /* Disable the layer */
+ atmel_hlcdc_layer_write_reg(&plane->layer, ATMEL_HLCDC_LAYER_CHDR,
+ ATMEL_HLCDC_LAYER_RST |
+ ATMEL_HLCDC_LAYER_A2Q |
+ ATMEL_HLCDC_LAYER_UPDATE);
+
+ /* Clear all pending interrupts */
+ atmel_hlcdc_layer_read_reg(&plane->layer, ATMEL_HLCDC_LAYER_ISR);
}
static void atmel_hlcdc_plane_destroy(struct drm_plane *p)
if (plane->base.fb)
drm_framebuffer_unreference(plane->base.fb);
- atmel_hlcdc_layer_cleanup(p->dev, &plane->layer);
-
drm_plane_cleanup(p);
- devm_kfree(p->dev->dev, plane);
}
static int atmel_hlcdc_plane_atomic_set_property(struct drm_plane *p,
}
static int atmel_hlcdc_plane_init_properties(struct atmel_hlcdc_plane *plane,
- const struct atmel_hlcdc_layer_desc *desc,
- struct atmel_hlcdc_plane_properties *props)
+ struct atmel_hlcdc_plane_properties *props)
{
- struct regmap *regmap = plane->layer.hlcdc->regmap;
+ const struct atmel_hlcdc_layer_desc *desc = plane->layer.desc;
if (desc->type == ATMEL_HLCDC_OVERLAY_LAYER ||
- desc->type == ATMEL_HLCDC_CURSOR_LAYER) {
+ desc->type == ATMEL_HLCDC_CURSOR_LAYER)
drm_object_attach_property(&plane->base.base,
props->alpha, 255);
- /* Set default alpha value */
- regmap_update_bits(regmap,
- desc->regs_offset +
- ATMEL_HLCDC_LAYER_GENERAL_CFG(&plane->layer),
- ATMEL_HLCDC_LAYER_GA_MASK,
- ATMEL_HLCDC_LAYER_GA_MASK);
- }
-
if (desc->layout.xstride && desc->layout.pstride) {
int ret;
* TODO: decare a "yuv-to-rgb-conv-factors" property to let
* userspace modify these factors (using a BLOB property ?).
*/
- regmap_write(regmap,
- desc->regs_offset +
- ATMEL_HLCDC_LAYER_CSC_CFG(&plane->layer, 0),
- 0x4c900091);
- regmap_write(regmap,
- desc->regs_offset +
- ATMEL_HLCDC_LAYER_CSC_CFG(&plane->layer, 1),
- 0x7a5f5090);
- regmap_write(regmap,
- desc->regs_offset +
- ATMEL_HLCDC_LAYER_CSC_CFG(&plane->layer, 2),
- 0x40040890);
+ atmel_hlcdc_layer_write_cfg(&plane->layer,
+ desc->layout.csc,
+ 0x4c900091);
+ atmel_hlcdc_layer_write_cfg(&plane->layer,
+ desc->layout.csc + 1,
+ 0x7a5f5090);
+ atmel_hlcdc_layer_write_cfg(&plane->layer,
+ desc->layout.csc + 2,
+ 0x40040890);
}
return 0;
}
+void atmel_hlcdc_plane_irq(struct atmel_hlcdc_plane *plane)
+{
+ const struct atmel_hlcdc_layer_desc *desc = plane->layer.desc;
+ u32 isr;
+
+ isr = atmel_hlcdc_layer_read_reg(&plane->layer, ATMEL_HLCDC_LAYER_ISR);
+
+ /*
+ * There's not much we can do in case of overrun except informing
+ * the user. However, we are in interrupt context here, hence the
+ * use of dev_dbg().
+ */
+ if (isr &
+ (ATMEL_HLCDC_LAYER_OVR_IRQ(0) | ATMEL_HLCDC_LAYER_OVR_IRQ(1) |
+ ATMEL_HLCDC_LAYER_OVR_IRQ(2)))
+ dev_dbg(plane->base.dev->dev, "overrun on plane %s\n",
+ desc->name);
+}
+
static struct drm_plane_helper_funcs atmel_hlcdc_layer_plane_helper_funcs = {
- .prepare_fb = atmel_hlcdc_plane_prepare_fb,
- .cleanup_fb = atmel_hlcdc_plane_cleanup_fb,
.atomic_check = atmel_hlcdc_plane_atomic_check,
.atomic_update = atmel_hlcdc_plane_atomic_update,
.atomic_disable = atmel_hlcdc_plane_atomic_disable,
};
+static int atmel_hlcdc_plane_alloc_dscrs(struct drm_plane *p,
+ struct atmel_hlcdc_plane_state *state)
+{
+ struct atmel_hlcdc_dc *dc = p->dev->dev_private;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(state->dscrs); i++) {
+ struct atmel_hlcdc_dma_channel_dscr *dscr;
+ dma_addr_t dscr_dma;
+
+ dscr = dma_pool_alloc(dc->dscrpool, GFP_KERNEL, &dscr_dma);
+ if (!dscr)
+ goto err;
+
+ dscr->addr = 0;
+ dscr->next = dscr_dma;
+ dscr->self = dscr_dma;
+ dscr->ctrl = ATMEL_HLCDC_LAYER_DFETCH;
+
+ state->dscrs[i] = dscr;
+ }
+
+ return 0;
+
+err:
+ for (i--; i >= 0; i--) {
+ dma_pool_free(dc->dscrpool, state->dscrs[i],
+ state->dscrs[i]->self);
+ }
+
+ return -ENOMEM;
+}
+
static void atmel_hlcdc_plane_reset(struct drm_plane *p)
{
struct atmel_hlcdc_plane_state *state;
state = kzalloc(sizeof(*state), GFP_KERNEL);
if (state) {
+ if (atmel_hlcdc_plane_alloc_dscrs(p, state)) {
+ kfree(state);
+ dev_err(p->dev->dev,
+ "Failed to allocate initial plane state\n");
+ return;
+ }
+
state->alpha = 255;
p->state = &state->base;
p->state->plane = p;
if (!copy)
return NULL;
- copy->disc_updated = false;
- copy->prepared = false;
+ if (atmel_hlcdc_plane_alloc_dscrs(p, copy)) {
+ kfree(copy);
+ return NULL;
+ }
if (copy->base.fb)
drm_framebuffer_reference(copy->base.fb);
return ©->base;
}
-static void atmel_hlcdc_plane_atomic_destroy_state(struct drm_plane *plane,
+static void atmel_hlcdc_plane_atomic_destroy_state(struct drm_plane *p,
struct drm_plane_state *s)
{
struct atmel_hlcdc_plane_state *state =
drm_plane_state_to_atmel_hlcdc_plane_state(s);
+ struct atmel_hlcdc_dc *dc = p->dev->dev_private;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(state->dscrs); i++) {
+ dma_pool_free(dc->dscrpool, state->dscrs[i],
+ state->dscrs[i]->self);
+ }
if (s->fb)
drm_framebuffer_unreference(s->fb);
.atomic_get_property = atmel_hlcdc_plane_atomic_get_property,
};
-static struct atmel_hlcdc_plane *
-atmel_hlcdc_plane_create(struct drm_device *dev,
- const struct atmel_hlcdc_layer_desc *desc,
- struct atmel_hlcdc_plane_properties *props)
+static int atmel_hlcdc_plane_create(struct drm_device *dev,
+ const struct atmel_hlcdc_layer_desc *desc,
+ struct atmel_hlcdc_plane_properties *props)
{
+ struct atmel_hlcdc_dc *dc = dev->dev_private;
struct atmel_hlcdc_plane *plane;
enum drm_plane_type type;
int ret;
plane = devm_kzalloc(dev->dev, sizeof(*plane), GFP_KERNEL);
if (!plane)
- return ERR_PTR(-ENOMEM);
+ return -ENOMEM;
- ret = atmel_hlcdc_layer_init(dev, &plane->layer, desc);
- if (ret)
- return ERR_PTR(ret);
+ atmel_hlcdc_layer_init(&plane->layer, desc, dc->hlcdc->regmap);
+ plane->properties = props;
if (desc->type == ATMEL_HLCDC_BASE_LAYER)
type = DRM_PLANE_TYPE_PRIMARY;
desc->formats->formats,
desc->formats->nformats, type, NULL);
if (ret)
- return ERR_PTR(ret);
+ return ret;
drm_plane_helper_add(&plane->base,
&atmel_hlcdc_layer_plane_helper_funcs);
/* Set default property values*/
- ret = atmel_hlcdc_plane_init_properties(plane, desc, props);
+ ret = atmel_hlcdc_plane_init_properties(plane, props);
if (ret)
- return ERR_PTR(ret);
+ return ret;
+
+ dc->layers[desc->id] = &plane->layer;
- return plane;
+ return 0;
}
static struct atmel_hlcdc_plane_properties *
return props;
}
-struct atmel_hlcdc_planes *
-atmel_hlcdc_create_planes(struct drm_device *dev)
+int atmel_hlcdc_create_planes(struct drm_device *dev)
{
struct atmel_hlcdc_dc *dc = dev->dev_private;
struct atmel_hlcdc_plane_properties *props;
- struct atmel_hlcdc_planes *planes;
const struct atmel_hlcdc_layer_desc *descs = dc->desc->layers;
int nlayers = dc->desc->nlayers;
- int i;
-
- planes = devm_kzalloc(dev->dev, sizeof(*planes), GFP_KERNEL);
- if (!planes)
- return ERR_PTR(-ENOMEM);
-
- for (i = 0; i < nlayers; i++) {
- if (descs[i].type == ATMEL_HLCDC_OVERLAY_LAYER)
- planes->noverlays++;
- }
-
- if (planes->noverlays) {
- planes->overlays = devm_kzalloc(dev->dev,
- planes->noverlays *
- sizeof(*planes->overlays),
- GFP_KERNEL);
- if (!planes->overlays)
- return ERR_PTR(-ENOMEM);
- }
+ int i, ret;
props = atmel_hlcdc_plane_create_properties(dev);
if (IS_ERR(props))
- return ERR_CAST(props);
+ return PTR_ERR(props);
- planes->noverlays = 0;
- for (i = 0; i < nlayers; i++) {
- struct atmel_hlcdc_plane *plane;
+ dc->dscrpool = dmam_pool_create("atmel-hlcdc-dscr", dev->dev,
+ sizeof(struct atmel_hlcdc_dma_channel_dscr),
+ sizeof(u64), 0);
+ if (!dc->dscrpool)
+ return -ENOMEM;
- if (descs[i].type == ATMEL_HLCDC_PP_LAYER)
+ for (i = 0; i < nlayers; i++) {
+ if (descs[i].type != ATMEL_HLCDC_BASE_LAYER &&
+ descs[i].type != ATMEL_HLCDC_OVERLAY_LAYER &&
+ descs[i].type != ATMEL_HLCDC_CURSOR_LAYER)
continue;
- plane = atmel_hlcdc_plane_create(dev, &descs[i], props);
- if (IS_ERR(plane))
- return ERR_CAST(plane);
-
- plane->properties = props;
-
- switch (descs[i].type) {
- case ATMEL_HLCDC_BASE_LAYER:
- if (planes->primary)
- return ERR_PTR(-EINVAL);
- planes->primary = plane;
- break;
-
- case ATMEL_HLCDC_OVERLAY_LAYER:
- planes->overlays[planes->noverlays++] = plane;
- break;
-
- case ATMEL_HLCDC_CURSOR_LAYER:
- if (planes->cursor)
- return ERR_PTR(-EINVAL);
- planes->cursor = plane;
- break;
-
- default:
- break;
- }
+ ret = atmel_hlcdc_plane_create(dev, &descs[i], props);
+ if (ret)
+ return ret;
}
- return planes;
+ return 0;
}
info->par = &bochs->fb.helper;
ret = bochs_framebuffer_init(bochs->dev, &bochs->fb.gfb, &mode_cmd, gobj);
- if (ret) {
- drm_fb_helper_release_fbi(helper);
+ if (ret)
return ret;
- }
bochs->fb.size = size;
info->fix.smem_start = 0;
info->fix.smem_len = size;
+ bochs->fb.initialized = true;
return 0;
}
DRM_DEBUG_DRIVER("\n");
drm_fb_helper_unregister_fbi(&bochs->fb.helper);
- drm_fb_helper_release_fbi(&bochs->fb.helper);
if (gfb->obj) {
drm_gem_object_unreference_unlocked(gfb->obj);
gfb->obj = NULL;
}
- drm_fb_helper_fini(&bochs->fb.helper);
drm_framebuffer_unregister_private(&gfb->base);
drm_framebuffer_cleanup(&gfb->base);
if (ret)
goto fini;
- bochs->fb.initialized = true;
return 0;
fini:
void bochs_fbdev_fini(struct bochs_device *bochs)
{
- if (!bochs->fb.initialized)
- return;
+ if (bochs->fb.initialized)
+ bochs_fbdev_destroy(bochs);
- bochs_fbdev_destroy(bochs);
+ drm_fb_helper_fini(&bochs->fb.helper);
bochs->fb.initialized = false;
}
help
Support for RGB to VGA DAC based bridges
-config DRM_DW_HDMI
- tristate
+config DRM_LVDS_ENCODER
+ tristate "Transparent parallel to LVDS encoder support"
+ depends on OF
select DRM_KMS_HELPER
-
-config DRM_DW_HDMI_AHB_AUDIO
- tristate "Synopsis Designware AHB Audio interface"
- depends on DRM_DW_HDMI && SND
- select SND_PCM
- select SND_PCM_ELD
- select SND_PCM_IEC958
+ select DRM_PANEL
help
- Support the AHB Audio interface which is part of the Synopsis
- Designware HDMI block. This is used in conjunction with
- the i.MX6 HDMI driver.
+ Support for transparent parallel to LVDS encoders that don't require
+ any configuration.
-config DRM_DW_HDMI_I2S_AUDIO
- tristate "Synopsis Designware I2S Audio interface"
- depends on SND_SOC
- depends on DRM_DW_HDMI
- select SND_SOC_HDMI_CODEC
- help
- Support the I2S Audio interface which is part of the Synopsis
- Designware HDMI block.
+config DRM_MEGACHIPS_STDPXXXX_GE_B850V3_FW
+ tristate "MegaChips stdp4028-ge-b850v3-fw and stdp2690-ge-b850v3-fw"
+ depends on OF
+ select DRM_KMS_HELPER
+ select DRM_PANEL
+ ---help---
+ This is a driver for the display bridges of
+ GE B850v3 that convert dual channel LVDS
+ to DP++. This is used with the i.MX6 imx-ldb
+ driver. You are likely to say N here.
config DRM_NXP_PTN3460
tristate "NXP PTN3460 DP/LVDS bridge"
source "drivers/gpu/drm/bridge/adv7511/Kconfig"
+source "drivers/gpu/drm/bridge/synopsys/Kconfig"
+
endmenu
obj-$(CONFIG_DRM_ANALOGIX_ANX78XX) += analogix-anx78xx.o
obj-$(CONFIG_DRM_DUMB_VGA_DAC) += dumb-vga-dac.o
-obj-$(CONFIG_DRM_DW_HDMI) += dw-hdmi.o
-obj-$(CONFIG_DRM_DW_HDMI_AHB_AUDIO) += dw-hdmi-ahb-audio.o
-obj-$(CONFIG_DRM_DW_HDMI_I2S_AUDIO) += dw-hdmi-i2s-audio.o
+obj-$(CONFIG_DRM_LVDS_ENCODER) += lvds-encoder.o
+obj-$(CONFIG_DRM_MEGACHIPS_STDPXXXX_GE_B850V3_FW) += megachips-stdpxxxx-ge-b850v3-fw.o
obj-$(CONFIG_DRM_NXP_PTN3460) += nxp-ptn3460.o
obj-$(CONFIG_DRM_PARADE_PS8622) += parade-ps8622.o
obj-$(CONFIG_DRM_SIL_SII8620) += sil-sii8620.o
obj-$(CONFIG_DRM_ANALOGIX_DP) += analogix/
obj-$(CONFIG_DRM_I2C_ADV7511) += adv7511/
obj-$(CONFIG_DRM_TI_TFP410) += ti-tfp410.o
+obj-y += synopsys/
EXPORT_SYMBOL_GPL(analogix_dp_resume);
#endif
+int analogix_dp_start_crc(struct drm_connector *connector)
+{
+ struct analogix_dp_device *dp = to_dp(connector);
+
+ if (!connector->state->crtc) {
+ DRM_ERROR("Connector %s doesn't currently have a CRTC.\n",
+ connector->name);
+ return -EINVAL;
+ }
+
+ return drm_dp_start_crc(&dp->aux, connector->state->crtc);
+}
+EXPORT_SYMBOL_GPL(analogix_dp_start_crc);
+
+int analogix_dp_stop_crc(struct drm_connector *connector)
+{
+ struct analogix_dp_device *dp = to_dp(connector);
+
+ return drm_dp_stop_crc(&dp->aux);
+}
+EXPORT_SYMBOL_GPL(analogix_dp_stop_crc);
+
MODULE_AUTHOR("Jingoo Han <jg1.han@samsung.com>");
MODULE_DESCRIPTION("Analogix DP Core Driver");
MODULE_LICENSE("GPL v2");
static const struct of_device_id dumb_vga_match[] = {
{ .compatible = "dumb-vga-dac" },
+ { .compatible = "adi,adv7123" },
{ .compatible = "ti,ths8135" },
{},
};
--- /dev/null
+/*
+ * Copyright (C) 2016 Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#include <drm/drmP.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_connector.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_encoder.h>
+#include <drm/drm_modeset_helper_vtables.h>
+#include <drm/drm_panel.h>
+
+#include <linux/of_graph.h>
+
+struct lvds_encoder {
+ struct device *dev;
+
+ struct drm_bridge bridge;
+ struct drm_connector connector;
+ struct drm_panel *panel;
+};
+
+static inline struct lvds_encoder *
+drm_bridge_to_lvds_encoder(struct drm_bridge *bridge)
+{
+ return container_of(bridge, struct lvds_encoder, bridge);
+}
+
+static inline struct lvds_encoder *
+drm_connector_to_lvds_encoder(struct drm_connector *connector)
+{
+ return container_of(connector, struct lvds_encoder, connector);
+}
+
+static int lvds_connector_get_modes(struct drm_connector *connector)
+{
+ struct lvds_encoder *lvds = drm_connector_to_lvds_encoder(connector);
+
+ return drm_panel_get_modes(lvds->panel);
+}
+
+static const struct drm_connector_helper_funcs lvds_connector_helper_funcs = {
+ .get_modes = lvds_connector_get_modes,
+};
+
+static const struct drm_connector_funcs lvds_connector_funcs = {
+ .dpms = drm_atomic_helper_connector_dpms,
+ .reset = drm_atomic_helper_connector_reset,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .destroy = drm_connector_cleanup,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+static int lvds_encoder_attach(struct drm_bridge *bridge)
+{
+ struct lvds_encoder *lvds = drm_bridge_to_lvds_encoder(bridge);
+ struct drm_connector *connector = &lvds->connector;
+ int ret;
+
+ if (!bridge->encoder) {
+ DRM_ERROR("Missing encoder\n");
+ return -ENODEV;
+ }
+
+ drm_connector_helper_add(connector, &lvds_connector_helper_funcs);
+
+ ret = drm_connector_init(bridge->dev, connector, &lvds_connector_funcs,
+ DRM_MODE_CONNECTOR_LVDS);
+ if (ret) {
+ DRM_ERROR("Failed to initialize connector\n");
+ return ret;
+ }
+
+ drm_mode_connector_attach_encoder(&lvds->connector, bridge->encoder);
+
+ ret = drm_panel_attach(lvds->panel, &lvds->connector);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static void lvds_encoder_detach(struct drm_bridge *bridge)
+{
+ struct lvds_encoder *lvds = drm_bridge_to_lvds_encoder(bridge);
+
+ drm_panel_detach(lvds->panel);
+}
+
+static void lvds_encoder_pre_enable(struct drm_bridge *bridge)
+{
+ struct lvds_encoder *lvds = drm_bridge_to_lvds_encoder(bridge);
+
+ drm_panel_prepare(lvds->panel);
+}
+
+static void lvds_encoder_enable(struct drm_bridge *bridge)
+{
+ struct lvds_encoder *lvds = drm_bridge_to_lvds_encoder(bridge);
+
+ drm_panel_enable(lvds->panel);
+}
+
+static void lvds_encoder_disable(struct drm_bridge *bridge)
+{
+ struct lvds_encoder *lvds = drm_bridge_to_lvds_encoder(bridge);
+
+ drm_panel_disable(lvds->panel);
+}
+
+static void lvds_encoder_post_disable(struct drm_bridge *bridge)
+{
+ struct lvds_encoder *lvds = drm_bridge_to_lvds_encoder(bridge);
+
+ drm_panel_unprepare(lvds->panel);
+}
+
+static const struct drm_bridge_funcs lvds_encoder_bridge_funcs = {
+ .attach = lvds_encoder_attach,
+ .detach = lvds_encoder_detach,
+ .pre_enable = lvds_encoder_pre_enable,
+ .enable = lvds_encoder_enable,
+ .disable = lvds_encoder_disable,
+ .post_disable = lvds_encoder_post_disable,
+};
+
+static int lvds_encoder_probe(struct platform_device *pdev)
+{
+ struct lvds_encoder *lvds;
+ struct device_node *port;
+ struct device_node *endpoint;
+ struct device_node *panel;
+
+ lvds = devm_kzalloc(&pdev->dev, sizeof(*lvds), GFP_KERNEL);
+ if (!lvds)
+ return -ENOMEM;
+
+ lvds->dev = &pdev->dev;
+ platform_set_drvdata(pdev, lvds);
+
+ lvds->bridge.funcs = &lvds_encoder_bridge_funcs;
+ lvds->bridge.of_node = pdev->dev.of_node;
+
+ /* Locate the panel DT node. */
+ port = of_graph_get_port_by_id(pdev->dev.of_node, 1);
+ if (!port) {
+ dev_dbg(&pdev->dev, "port 1 not found\n");
+ return -ENXIO;
+ }
+
+ endpoint = of_get_child_by_name(port, "endpoint");
+ of_node_put(port);
+ if (!endpoint) {
+ dev_dbg(&pdev->dev, "no endpoint for port 1\n");
+ return -ENXIO;
+ }
+
+ panel = of_graph_get_remote_port_parent(endpoint);
+ of_node_put(endpoint);
+ if (!panel) {
+ dev_dbg(&pdev->dev, "no remote endpoint for port 1\n");
+ return -ENXIO;
+ }
+
+ lvds->panel = of_drm_find_panel(panel);
+ of_node_put(panel);
+ if (!lvds->panel) {
+ dev_dbg(&pdev->dev, "panel not found, deferring probe\n");
+ return -EPROBE_DEFER;
+ }
+
+ /* Register the bridge. */
+ return drm_bridge_add(&lvds->bridge);
+}
+
+static int lvds_encoder_remove(struct platform_device *pdev)
+{
+ struct lvds_encoder *encoder = platform_get_drvdata(pdev);
+
+ drm_bridge_remove(&encoder->bridge);
+
+ return 0;
+}
+
+static const struct of_device_id lvds_encoder_match[] = {
+ { .compatible = "lvds-encoder" },
+ { .compatible = "thine,thc63lvdm83d" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, lvds_encoder_match);
+
+static struct platform_driver lvds_encoder_driver = {
+ .probe = lvds_encoder_probe,
+ .remove = lvds_encoder_remove,
+ .driver = {
+ .name = "lvds-encoder",
+ .of_match_table = lvds_encoder_match,
+ },
+};
+module_platform_driver(lvds_encoder_driver);
+
+MODULE_AUTHOR("Laurent Pinchart <laurent.pinchart@ideasonboard.com>");
+MODULE_DESCRIPTION("Transparent parallel to LVDS encoder");
+MODULE_LICENSE("GPL");
--- /dev/null
+/*
+ * Driver for MegaChips STDP4028 with GE B850v3 firmware (LVDS-DP)
+ * Driver for MegaChips STDP2690 with GE B850v3 firmware (DP-DP++)
+
+ * Copyright (c) 2017, Collabora Ltd.
+ * Copyright (c) 2017, General Electric Company
+
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+ * This driver creates a drm_bridge and a drm_connector for the LVDS to DP++
+ * display bridge of the GE B850v3. There are two physical bridges on the video
+ * signal pipeline: a STDP4028(LVDS to DP) and a STDP2690(DP to DP++). The
+ * physical bridges are automatically configured by the input video signal, and
+ * the driver has no access to the video processing pipeline. The driver is
+ * only needed to read EDID from the STDP2690 and to handle HPD events from the
+ * STDP4028. The driver communicates with both bridges over i2c. The video
+ * signal pipeline is as follows:
+ *
+ * Host -> LVDS|--(STDP4028)--|DP -> DP|--(STDP2690)--|DP++ -> Video output
+ *
+ */
+
+#include <linux/gpio.h>
+#include <linux/i2c.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <drm/drm_atomic.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_edid.h>
+#include <drm/drmP.h>
+
+#define EDID_EXT_BLOCK_CNT 0x7E
+
+#define STDP4028_IRQ_OUT_CONF_REG 0x02
+#define STDP4028_DPTX_IRQ_EN_REG 0x3C
+#define STDP4028_DPTX_IRQ_STS_REG 0x3D
+#define STDP4028_DPTX_STS_REG 0x3E
+
+#define STDP4028_DPTX_DP_IRQ_EN 0x1000
+
+#define STDP4028_DPTX_HOTPLUG_IRQ_EN 0x0400
+#define STDP4028_DPTX_LINK_CH_IRQ_EN 0x2000
+#define STDP4028_DPTX_IRQ_CONFIG \
+ (STDP4028_DPTX_LINK_CH_IRQ_EN | STDP4028_DPTX_HOTPLUG_IRQ_EN)
+
+#define STDP4028_DPTX_HOTPLUG_STS 0x0200
+#define STDP4028_DPTX_LINK_STS 0x1000
+#define STDP4028_CON_STATE_CONNECTED \
+ (STDP4028_DPTX_HOTPLUG_STS | STDP4028_DPTX_LINK_STS)
+
+#define STDP4028_DPTX_HOTPLUG_CH_STS 0x0400
+#define STDP4028_DPTX_LINK_CH_STS 0x2000
+#define STDP4028_DPTX_IRQ_CLEAR \
+ (STDP4028_DPTX_LINK_CH_STS | STDP4028_DPTX_HOTPLUG_CH_STS)
+
+static DEFINE_MUTEX(ge_b850v3_lvds_dev_mutex);
+
+struct ge_b850v3_lvds {
+ struct drm_connector connector;
+ struct drm_bridge bridge;
+ struct i2c_client *stdp4028_i2c;
+ struct i2c_client *stdp2690_i2c;
+ struct edid *edid;
+};
+
+static struct ge_b850v3_lvds *ge_b850v3_lvds_ptr;
+
+static u8 *stdp2690_get_edid(struct i2c_client *client)
+{
+ struct i2c_adapter *adapter = client->adapter;
+ unsigned char start = 0x00;
+ unsigned int total_size;
+ u8 *block = kmalloc(EDID_LENGTH, GFP_KERNEL);
+
+ struct i2c_msg msgs[] = {
+ {
+ .addr = client->addr,
+ .flags = 0,
+ .len = 1,
+ .buf = &start,
+ }, {
+ .addr = client->addr,
+ .flags = I2C_M_RD,
+ .len = EDID_LENGTH,
+ .buf = block,
+ }
+ };
+
+ if (!block)
+ return NULL;
+
+ if (i2c_transfer(adapter, msgs, 2) != 2) {
+ DRM_ERROR("Unable to read EDID.\n");
+ goto err;
+ }
+
+ if (!drm_edid_block_valid(block, 0, false, NULL)) {
+ DRM_ERROR("Invalid EDID data\n");
+ goto err;
+ }
+
+ total_size = (block[EDID_EXT_BLOCK_CNT] + 1) * EDID_LENGTH;
+ if (total_size > EDID_LENGTH) {
+ kfree(block);
+ block = kmalloc(total_size, GFP_KERNEL);
+ if (!block)
+ return NULL;
+
+ /* Yes, read the entire buffer, and do not skip the first
+ * EDID_LENGTH bytes.
+ */
+ start = 0x00;
+ msgs[1].len = total_size;
+ msgs[1].buf = block;
+
+ if (i2c_transfer(adapter, msgs, 2) != 2) {
+ DRM_ERROR("Unable to read EDID extension blocks.\n");
+ goto err;
+ }
+ if (!drm_edid_block_valid(block, 1, false, NULL)) {
+ DRM_ERROR("Invalid EDID data\n");
+ goto err;
+ }
+ }
+
+ return block;
+
+err:
+ kfree(block);
+ return NULL;
+}
+
+static int ge_b850v3_lvds_get_modes(struct drm_connector *connector)
+{
+ struct i2c_client *client;
+ int num_modes = 0;
+
+ client = ge_b850v3_lvds_ptr->stdp2690_i2c;
+
+ kfree(ge_b850v3_lvds_ptr->edid);
+ ge_b850v3_lvds_ptr->edid = (struct edid *)stdp2690_get_edid(client);
+
+ if (ge_b850v3_lvds_ptr->edid) {
+ drm_mode_connector_update_edid_property(connector,
+ ge_b850v3_lvds_ptr->edid);
+ num_modes = drm_add_edid_modes(connector,
+ ge_b850v3_lvds_ptr->edid);
+ }
+
+ return num_modes;
+}
+
+static enum drm_mode_status ge_b850v3_lvds_mode_valid(
+ struct drm_connector *connector, struct drm_display_mode *mode)
+{
+ return MODE_OK;
+}
+
+static const struct
+drm_connector_helper_funcs ge_b850v3_lvds_connector_helper_funcs = {
+ .get_modes = ge_b850v3_lvds_get_modes,
+ .mode_valid = ge_b850v3_lvds_mode_valid,
+};
+
+static enum drm_connector_status ge_b850v3_lvds_detect(
+ struct drm_connector *connector, bool force)
+{
+ struct i2c_client *stdp4028_i2c =
+ ge_b850v3_lvds_ptr->stdp4028_i2c;
+ s32 link_state;
+
+ link_state = i2c_smbus_read_word_data(stdp4028_i2c,
+ STDP4028_DPTX_STS_REG);
+
+ if (link_state == STDP4028_CON_STATE_CONNECTED)
+ return connector_status_connected;
+
+ if (link_state == 0)
+ return connector_status_disconnected;
+
+ return connector_status_unknown;
+}
+
+static const struct drm_connector_funcs ge_b850v3_lvds_connector_funcs = {
+ .dpms = drm_atomic_helper_connector_dpms,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .detect = ge_b850v3_lvds_detect,
+ .destroy = drm_connector_cleanup,
+ .reset = drm_atomic_helper_connector_reset,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+static irqreturn_t ge_b850v3_lvds_irq_handler(int irq, void *dev_id)
+{
+ struct i2c_client *stdp4028_i2c
+ = ge_b850v3_lvds_ptr->stdp4028_i2c;
+
+ i2c_smbus_write_word_data(stdp4028_i2c,
+ STDP4028_DPTX_IRQ_STS_REG,
+ STDP4028_DPTX_IRQ_CLEAR);
+
+ if (ge_b850v3_lvds_ptr->connector.dev)
+ drm_kms_helper_hotplug_event(ge_b850v3_lvds_ptr->connector.dev);
+
+ return IRQ_HANDLED;
+}
+
+static int ge_b850v3_lvds_attach(struct drm_bridge *bridge)
+{
+ struct drm_connector *connector = &ge_b850v3_lvds_ptr->connector;
+ struct i2c_client *stdp4028_i2c
+ = ge_b850v3_lvds_ptr->stdp4028_i2c;
+ int ret;
+
+ if (!bridge->encoder) {
+ DRM_ERROR("Parent encoder object not found");
+ return -ENODEV;
+ }
+
+ connector->polled = DRM_CONNECTOR_POLL_HPD;
+
+ drm_connector_helper_add(connector,
+ &ge_b850v3_lvds_connector_helper_funcs);
+
+ ret = drm_connector_init(bridge->dev, connector,
+ &ge_b850v3_lvds_connector_funcs,
+ DRM_MODE_CONNECTOR_DisplayPort);
+ if (ret) {
+ DRM_ERROR("Failed to initialize connector with drm\n");
+ return ret;
+ }
+
+ ret = drm_mode_connector_attach_encoder(connector, bridge->encoder);
+ if (ret)
+ return ret;
+
+ /* Configures the bridge to re-enable interrupts after each ack. */
+ i2c_smbus_write_word_data(stdp4028_i2c,
+ STDP4028_IRQ_OUT_CONF_REG,
+ STDP4028_DPTX_DP_IRQ_EN);
+
+ /* Enable interrupts */
+ i2c_smbus_write_word_data(stdp4028_i2c,
+ STDP4028_DPTX_IRQ_EN_REG,
+ STDP4028_DPTX_IRQ_CONFIG);
+
+ return 0;
+}
+
+static const struct drm_bridge_funcs ge_b850v3_lvds_funcs = {
+ .attach = ge_b850v3_lvds_attach,
+};
+
+static int ge_b850v3_lvds_init(struct device *dev)
+{
+ mutex_lock(&ge_b850v3_lvds_dev_mutex);
+
+ if (ge_b850v3_lvds_ptr)
+ goto success;
+
+ ge_b850v3_lvds_ptr = devm_kzalloc(dev,
+ sizeof(*ge_b850v3_lvds_ptr),
+ GFP_KERNEL);
+
+ if (!ge_b850v3_lvds_ptr) {
+ mutex_unlock(&ge_b850v3_lvds_dev_mutex);
+ return -ENOMEM;
+ }
+
+ ge_b850v3_lvds_ptr->bridge.funcs = &ge_b850v3_lvds_funcs;
+ ge_b850v3_lvds_ptr->bridge.of_node = dev->of_node;
+ drm_bridge_add(&ge_b850v3_lvds_ptr->bridge);
+
+success:
+ mutex_unlock(&ge_b850v3_lvds_dev_mutex);
+ return 0;
+}
+
+static void ge_b850v3_lvds_remove(void)
+{
+ mutex_lock(&ge_b850v3_lvds_dev_mutex);
+ /*
+ * This check is to avoid both the drivers
+ * removing the bridge in their remove() function
+ */
+ if (!ge_b850v3_lvds_ptr)
+ goto out;
+
+ drm_bridge_remove(&ge_b850v3_lvds_ptr->bridge);
+
+ kfree(ge_b850v3_lvds_ptr->edid);
+
+ ge_b850v3_lvds_ptr = NULL;
+out:
+ mutex_unlock(&ge_b850v3_lvds_dev_mutex);
+}
+
+static int stdp4028_ge_b850v3_fw_probe(struct i2c_client *stdp4028_i2c,
+ const struct i2c_device_id *id)
+{
+ struct device *dev = &stdp4028_i2c->dev;
+
+ ge_b850v3_lvds_init(dev);
+
+ ge_b850v3_lvds_ptr->stdp4028_i2c = stdp4028_i2c;
+ i2c_set_clientdata(stdp4028_i2c, ge_b850v3_lvds_ptr);
+
+ /* Clear pending interrupts since power up. */
+ i2c_smbus_write_word_data(stdp4028_i2c,
+ STDP4028_DPTX_IRQ_STS_REG,
+ STDP4028_DPTX_IRQ_CLEAR);
+
+ if (!stdp4028_i2c->irq)
+ return 0;
+
+ return devm_request_threaded_irq(&stdp4028_i2c->dev,
+ stdp4028_i2c->irq, NULL,
+ ge_b850v3_lvds_irq_handler,
+ IRQF_TRIGGER_HIGH | IRQF_ONESHOT,
+ "ge-b850v3-lvds-dp", ge_b850v3_lvds_ptr);
+}
+
+static int stdp4028_ge_b850v3_fw_remove(struct i2c_client *stdp4028_i2c)
+{
+ ge_b850v3_lvds_remove();
+
+ return 0;
+}
+
+static const struct i2c_device_id stdp4028_ge_b850v3_fw_i2c_table[] = {
+ {"stdp4028_ge_fw", 0},
+ {},
+};
+MODULE_DEVICE_TABLE(i2c, stdp4028_ge_b850v3_fw_i2c_table);
+
+static const struct of_device_id stdp4028_ge_b850v3_fw_match[] = {
+ { .compatible = "megachips,stdp4028-ge-b850v3-fw" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, stdp4028_ge_b850v3_fw_match);
+
+static struct i2c_driver stdp4028_ge_b850v3_fw_driver = {
+ .id_table = stdp4028_ge_b850v3_fw_i2c_table,
+ .probe = stdp4028_ge_b850v3_fw_probe,
+ .remove = stdp4028_ge_b850v3_fw_remove,
+ .driver = {
+ .name = "stdp4028-ge-b850v3-fw",
+ .of_match_table = stdp4028_ge_b850v3_fw_match,
+ },
+};
+
+static int stdp2690_ge_b850v3_fw_probe(struct i2c_client *stdp2690_i2c,
+ const struct i2c_device_id *id)
+{
+ struct device *dev = &stdp2690_i2c->dev;
+
+ ge_b850v3_lvds_init(dev);
+
+ ge_b850v3_lvds_ptr->stdp2690_i2c = stdp2690_i2c;
+ i2c_set_clientdata(stdp2690_i2c, ge_b850v3_lvds_ptr);
+
+ return 0;
+}
+
+static int stdp2690_ge_b850v3_fw_remove(struct i2c_client *stdp2690_i2c)
+{
+ ge_b850v3_lvds_remove();
+
+ return 0;
+}
+
+static const struct i2c_device_id stdp2690_ge_b850v3_fw_i2c_table[] = {
+ {"stdp2690_ge_fw", 0},
+ {},
+};
+MODULE_DEVICE_TABLE(i2c, stdp2690_ge_b850v3_fw_i2c_table);
+
+static const struct of_device_id stdp2690_ge_b850v3_fw_match[] = {
+ { .compatible = "megachips,stdp2690-ge-b850v3-fw" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, stdp2690_ge_b850v3_fw_match);
+
+static struct i2c_driver stdp2690_ge_b850v3_fw_driver = {
+ .id_table = stdp2690_ge_b850v3_fw_i2c_table,
+ .probe = stdp2690_ge_b850v3_fw_probe,
+ .remove = stdp2690_ge_b850v3_fw_remove,
+ .driver = {
+ .name = "stdp2690-ge-b850v3-fw",
+ .of_match_table = stdp2690_ge_b850v3_fw_match,
+ },
+};
+
+static int __init stdpxxxx_ge_b850v3_init(void)
+{
+ int ret;
+
+ ret = i2c_add_driver(&stdp4028_ge_b850v3_fw_driver);
+ if (ret)
+ return ret;
+
+ return i2c_add_driver(&stdp2690_ge_b850v3_fw_driver);
+}
+module_init(stdpxxxx_ge_b850v3_init);
+
+static void __exit stdpxxxx_ge_b850v3_exit(void)
+{
+ i2c_del_driver(&stdp2690_ge_b850v3_fw_driver);
+ i2c_del_driver(&stdp4028_ge_b850v3_fw_driver);
+}
+module_exit(stdpxxxx_ge_b850v3_exit);
+
+MODULE_AUTHOR("Peter Senna Tschudin <peter.senna@collabora.com>");
+MODULE_AUTHOR("Martyn Welch <martyn.welch@collabora.co.uk>");
+MODULE_DESCRIPTION("GE LVDS to DP++ display bridge)");
+MODULE_LICENSE("GPL v2");
sii8620_irq_thread,
IRQF_TRIGGER_HIGH | IRQF_ONESHOT,
"sii8620", ctx);
+ if (ret < 0) {
+ dev_err(dev, "failed to install IRQ handler\n");
+ return ret;
+ }
ctx->gpio_reset = devm_gpiod_get(dev, "reset", GPIOD_OUT_HIGH);
if (IS_ERR(ctx->gpio_reset)) {
--- /dev/null
+config DRM_DW_HDMI
+ tristate
+ select DRM_KMS_HELPER
+
+config DRM_DW_HDMI_AHB_AUDIO
+ tristate "Synopsys Designware AHB Audio interface"
+ depends on DRM_DW_HDMI && SND
+ select SND_PCM
+ select SND_PCM_ELD
+ select SND_PCM_IEC958
+ help
+ Support the AHB Audio interface which is part of the Synopsys
+ Designware HDMI block. This is used in conjunction with
+ the i.MX6 HDMI driver.
+
+config DRM_DW_HDMI_I2S_AUDIO
+ tristate "Synopsys Designware I2S Audio interface"
+ depends on SND_SOC
+ depends on DRM_DW_HDMI
+ select SND_SOC_HDMI_CODEC
+ help
+ Support the I2S Audio interface which is part of the Synopsys
+ Designware HDMI block.
--- /dev/null
+#ccflags-y := -Iinclude/drm
+
+obj-$(CONFIG_DRM_DW_HDMI) += dw-hdmi.o
+obj-$(CONFIG_DRM_DW_HDMI_AHB_AUDIO) += dw-hdmi-ahb-audio.o
+obj-$(CONFIG_DRM_DW_HDMI_I2S_AUDIO) += dw-hdmi-i2s-audio.o
#include <linux/hdmi.h>
#include <linux/mutex.h>
#include <linux/of_device.h>
+#include <linux/regmap.h>
#include <linux/spinlock.h>
#include <drm/drm_of.h>
#include "dw-hdmi.h"
#include "dw-hdmi-audio.h"
+#define DDC_SEGMENT_ADDR 0x30
#define HDMI_EDID_LEN 512
#define RGB 0
u8 slave_reg;
bool is_regaddr;
+ bool is_segment;
};
struct dw_hdmi_phy_data {
enum dw_hdmi_phy_type type;
const char *name;
+ unsigned int gen;
bool has_svsret;
+ int (*configure)(struct dw_hdmi *hdmi,
+ const struct dw_hdmi_plat_data *pdata,
+ unsigned long mpixelclock);
};
struct dw_hdmi {
struct drm_connector connector;
struct drm_bridge bridge;
- enum dw_hdmi_devtype dev_type;
unsigned int version;
struct platform_device *audio;
u8 edid[HDMI_EDID_LEN];
bool cable_plugin;
- const struct dw_hdmi_phy_data *phy;
- bool phy_enabled;
+ struct {
+ const struct dw_hdmi_phy_ops *ops;
+ const char *name;
+ void *data;
+ bool enabled;
+ } phy;
struct drm_display_mode previous_mode;
unsigned int audio_n;
bool audio_enable;
- void (*write)(struct dw_hdmi *hdmi, u8 val, int offset);
- u8 (*read)(struct dw_hdmi *hdmi, int offset);
+ unsigned int reg_shift;
+ struct regmap *regm;
};
#define HDMI_IH_PHY_STAT0_RX_SENSE \
(HDMI_PHY_RX_SENSE0 | HDMI_PHY_RX_SENSE1 | \
HDMI_PHY_RX_SENSE2 | HDMI_PHY_RX_SENSE3)
-static void dw_hdmi_writel(struct dw_hdmi *hdmi, u8 val, int offset)
-{
- writel(val, hdmi->regs + (offset << 2));
-}
-
-static u8 dw_hdmi_readl(struct dw_hdmi *hdmi, int offset)
-{
- return readl(hdmi->regs + (offset << 2));
-}
-
-static void dw_hdmi_writeb(struct dw_hdmi *hdmi, u8 val, int offset)
-{
- writeb(val, hdmi->regs + offset);
-}
-
-static u8 dw_hdmi_readb(struct dw_hdmi *hdmi, int offset)
-{
- return readb(hdmi->regs + offset);
-}
-
static inline void hdmi_writeb(struct dw_hdmi *hdmi, u8 val, int offset)
{
- hdmi->write(hdmi, val, offset);
+ regmap_write(hdmi->regm, offset << hdmi->reg_shift, val);
}
static inline u8 hdmi_readb(struct dw_hdmi *hdmi, int offset)
{
- return hdmi->read(hdmi, offset);
+ unsigned int val = 0;
+
+ regmap_read(hdmi->regm, offset << hdmi->reg_shift, &val);
+
+ return val;
}
static void hdmi_modb(struct dw_hdmi *hdmi, u8 data, u8 mask, unsigned reg)
{
- u8 val = hdmi_readb(hdmi, reg) & ~mask;
-
- val |= data & mask;
- hdmi_writeb(hdmi, val, reg);
+ regmap_update_bits(hdmi->regm, reg << hdmi->reg_shift, mask, data);
}
static void hdmi_mask_writeb(struct dw_hdmi *hdmi, u8 data, unsigned int reg,
reinit_completion(&i2c->cmp);
hdmi_writeb(hdmi, i2c->slave_reg++, HDMI_I2CM_ADDRESS);
- hdmi_writeb(hdmi, HDMI_I2CM_OPERATION_READ,
- HDMI_I2CM_OPERATION);
+ if (i2c->is_segment)
+ hdmi_writeb(hdmi, HDMI_I2CM_OPERATION_READ_EXT,
+ HDMI_I2CM_OPERATION);
+ else
+ hdmi_writeb(hdmi, HDMI_I2CM_OPERATION_READ,
+ HDMI_I2CM_OPERATION);
stat = wait_for_completion_timeout(&i2c->cmp, HZ / 10);
if (!stat)
*buf++ = hdmi_readb(hdmi, HDMI_I2CM_DATAI);
}
+ i2c->is_segment = false;
return 0;
}
dev_dbg(hdmi->dev, "xfer: num: %d, addr: %#x\n", num, addr);
for (i = 0; i < num; i++) {
- if (msgs[i].addr != addr) {
- dev_warn(hdmi->dev,
- "unsupported transfer, changed slave address\n");
- return -EOPNOTSUPP;
- }
-
if (msgs[i].len == 0) {
dev_dbg(hdmi->dev,
"unsupported transfer %d/%d, no data\n",
/* Set slave device register address on transfer */
i2c->is_regaddr = false;
+ /* Set segment pointer for I2C extended read mode operation */
+ i2c->is_segment = false;
+
for (i = 0; i < num; i++) {
dev_dbg(hdmi->dev, "xfer: num: %d/%d, len: %d, flags: %#x\n",
i + 1, num, msgs[i].len, msgs[i].flags);
-
- if (msgs[i].flags & I2C_M_RD)
- ret = dw_hdmi_i2c_read(hdmi, msgs[i].buf, msgs[i].len);
- else
- ret = dw_hdmi_i2c_write(hdmi, msgs[i].buf, msgs[i].len);
-
+ if (msgs[i].addr == DDC_SEGMENT_ADDR && msgs[i].len == 1) {
+ i2c->is_segment = true;
+ hdmi_writeb(hdmi, DDC_SEGMENT_ADDR, HDMI_I2CM_SEGADDR);
+ hdmi_writeb(hdmi, *msgs[i].buf, HDMI_I2CM_SEGPTR);
+ } else {
+ if (msgs[i].flags & I2C_M_RD)
+ ret = dw_hdmi_i2c_read(hdmi, msgs[i].buf,
+ msgs[i].len);
+ else
+ ret = dw_hdmi_i2c_write(hdmi, msgs[i].buf,
+ msgs[i].len);
+ }
if (ret < 0)
break;
}
HDMI_VP_CONF);
}
+/* -----------------------------------------------------------------------------
+ * Synopsys PHY Handling
+ */
+
static inline void hdmi_phy_test_clear(struct dw_hdmi *hdmi,
unsigned char bit)
{
HDMI_PHY_TST0_TSTCLR_MASK, HDMI_PHY_TST0);
}
-static inline void hdmi_phy_test_enable(struct dw_hdmi *hdmi,
- unsigned char bit)
-{
- hdmi_modb(hdmi, bit << HDMI_PHY_TST0_TSTEN_OFFSET,
- HDMI_PHY_TST0_TSTEN_MASK, HDMI_PHY_TST0);
-}
-
-static inline void hdmi_phy_test_clock(struct dw_hdmi *hdmi,
- unsigned char bit)
-{
- hdmi_modb(hdmi, bit << HDMI_PHY_TST0_TSTCLK_OFFSET,
- HDMI_PHY_TST0_TSTCLK_MASK, HDMI_PHY_TST0);
-}
-
-static inline void hdmi_phy_test_din(struct dw_hdmi *hdmi,
- unsigned char bit)
-{
- hdmi_writeb(hdmi, bit, HDMI_PHY_TST1);
-}
-
-static inline void hdmi_phy_test_dout(struct dw_hdmi *hdmi,
- unsigned char bit)
-{
- hdmi_writeb(hdmi, bit, HDMI_PHY_TST2);
-}
-
static bool hdmi_phy_wait_i2c_done(struct dw_hdmi *hdmi, int msec)
{
u32 val;
return true;
}
-static void hdmi_phy_i2c_write(struct dw_hdmi *hdmi, unsigned short data,
- unsigned char addr)
+void dw_hdmi_phy_i2c_write(struct dw_hdmi *hdmi, unsigned short data,
+ unsigned char addr)
{
hdmi_writeb(hdmi, 0xFF, HDMI_IH_I2CMPHY_STAT0);
hdmi_writeb(hdmi, addr, HDMI_PHY_I2CM_ADDRESS_ADDR);
HDMI_PHY_I2CM_OPERATION_ADDR);
hdmi_phy_wait_i2c_done(hdmi, 1000);
}
+EXPORT_SYMBOL_GPL(dw_hdmi_phy_i2c_write);
static void dw_hdmi_phy_enable_powerdown(struct dw_hdmi *hdmi, bool enable)
{
HDMI_PHY_CONF0_SELDIPIF_MASK);
}
-static int hdmi_phy_configure(struct dw_hdmi *hdmi, int cscon)
+static void dw_hdmi_phy_power_off(struct dw_hdmi *hdmi)
+{
+ const struct dw_hdmi_phy_data *phy = hdmi->phy.data;
+ unsigned int i;
+ u16 val;
+
+ if (phy->gen == 1) {
+ dw_hdmi_phy_enable_tmds(hdmi, 0);
+ dw_hdmi_phy_enable_powerdown(hdmi, true);
+ return;
+ }
+
+ dw_hdmi_phy_gen2_txpwron(hdmi, 0);
+
+ /*
+ * Wait for TX_PHY_LOCK to be deasserted to indicate that the PHY went
+ * to low power mode.
+ */
+ for (i = 0; i < 5; ++i) {
+ val = hdmi_readb(hdmi, HDMI_PHY_STAT0);
+ if (!(val & HDMI_PHY_TX_PHY_LOCK))
+ break;
+
+ usleep_range(1000, 2000);
+ }
+
+ if (val & HDMI_PHY_TX_PHY_LOCK)
+ dev_warn(hdmi->dev, "PHY failed to power down\n");
+ else
+ dev_dbg(hdmi->dev, "PHY powered down in %u iterations\n", i);
+
+ dw_hdmi_phy_gen2_pddq(hdmi, 1);
+}
+
+static int dw_hdmi_phy_power_on(struct dw_hdmi *hdmi)
+{
+ const struct dw_hdmi_phy_data *phy = hdmi->phy.data;
+ unsigned int i;
+ u8 val;
+
+ if (phy->gen == 1) {
+ dw_hdmi_phy_enable_powerdown(hdmi, false);
+
+ /* Toggle TMDS enable. */
+ dw_hdmi_phy_enable_tmds(hdmi, 0);
+ dw_hdmi_phy_enable_tmds(hdmi, 1);
+ return 0;
+ }
+
+ dw_hdmi_phy_gen2_txpwron(hdmi, 1);
+ dw_hdmi_phy_gen2_pddq(hdmi, 0);
+
+ /* Wait for PHY PLL lock */
+ for (i = 0; i < 5; ++i) {
+ val = hdmi_readb(hdmi, HDMI_PHY_STAT0) & HDMI_PHY_TX_PHY_LOCK;
+ if (val)
+ break;
+
+ usleep_range(1000, 2000);
+ }
+
+ if (!val) {
+ dev_err(hdmi->dev, "PHY PLL failed to lock\n");
+ return -ETIMEDOUT;
+ }
+
+ dev_dbg(hdmi->dev, "PHY PLL locked %u iterations\n", i);
+ return 0;
+}
+
+/*
+ * PHY configuration function for the DWC HDMI 3D TX PHY. Based on the available
+ * information the DWC MHL PHY has the same register layout and is thus also
+ * supported by this function.
+ */
+static int hdmi_phy_configure_dwc_hdmi_3d_tx(struct dw_hdmi *hdmi,
+ const struct dw_hdmi_plat_data *pdata,
+ unsigned long mpixelclock)
{
- u8 val, msec;
- const struct dw_hdmi_plat_data *pdata = hdmi->plat_data;
const struct dw_hdmi_mpll_config *mpll_config = pdata->mpll_cfg;
const struct dw_hdmi_curr_ctrl *curr_ctrl = pdata->cur_ctr;
const struct dw_hdmi_phy_config *phy_config = pdata->phy_config;
/* PLL/MPLL Cfg - always match on final entry */
for (; mpll_config->mpixelclock != ~0UL; mpll_config++)
- if (hdmi->hdmi_data.video_mode.mpixelclock <=
- mpll_config->mpixelclock)
+ if (mpixelclock <= mpll_config->mpixelclock)
break;
for (; curr_ctrl->mpixelclock != ~0UL; curr_ctrl++)
- if (hdmi->hdmi_data.video_mode.mpixelclock <=
- curr_ctrl->mpixelclock)
+ if (mpixelclock <= curr_ctrl->mpixelclock)
break;
for (; phy_config->mpixelclock != ~0UL; phy_config++)
- if (hdmi->hdmi_data.video_mode.mpixelclock <=
- phy_config->mpixelclock)
+ if (mpixelclock <= phy_config->mpixelclock)
break;
if (mpll_config->mpixelclock == ~0UL ||
curr_ctrl->mpixelclock == ~0UL ||
- phy_config->mpixelclock == ~0UL) {
- dev_err(hdmi->dev, "Pixel clock %d - unsupported by HDMI\n",
- hdmi->hdmi_data.video_mode.mpixelclock);
+ phy_config->mpixelclock == ~0UL)
return -EINVAL;
- }
- /* Enable csc path */
- if (cscon)
- val = HDMI_MC_FLOWCTRL_FEED_THROUGH_OFF_CSC_IN_PATH;
- else
- val = HDMI_MC_FLOWCTRL_FEED_THROUGH_OFF_CSC_BYPASS;
+ dw_hdmi_phy_i2c_write(hdmi, mpll_config->res[0].cpce,
+ HDMI_3D_TX_PHY_CPCE_CTRL);
+ dw_hdmi_phy_i2c_write(hdmi, mpll_config->res[0].gmp,
+ HDMI_3D_TX_PHY_GMPCTRL);
+ dw_hdmi_phy_i2c_write(hdmi, curr_ctrl->curr[0],
+ HDMI_3D_TX_PHY_CURRCTRL);
- hdmi_writeb(hdmi, val, HDMI_MC_FLOWCTRL);
+ dw_hdmi_phy_i2c_write(hdmi, 0, HDMI_3D_TX_PHY_PLLPHBYCTRL);
+ dw_hdmi_phy_i2c_write(hdmi, HDMI_3D_TX_PHY_MSM_CTRL_CKO_SEL_FB_CLK,
+ HDMI_3D_TX_PHY_MSM_CTRL);
- /* gen2 tx power off */
- dw_hdmi_phy_gen2_txpwron(hdmi, 0);
+ dw_hdmi_phy_i2c_write(hdmi, phy_config->term, HDMI_3D_TX_PHY_TXTERM);
+ dw_hdmi_phy_i2c_write(hdmi, phy_config->sym_ctr,
+ HDMI_3D_TX_PHY_CKSYMTXCTRL);
+ dw_hdmi_phy_i2c_write(hdmi, phy_config->vlev_ctr,
+ HDMI_3D_TX_PHY_VLEVCTRL);
- /* gen2 pddq */
- dw_hdmi_phy_gen2_pddq(hdmi, 1);
+ /* Override and disable clock termination. */
+ dw_hdmi_phy_i2c_write(hdmi, HDMI_3D_TX_PHY_CKCALCTRL_OVERRIDE,
+ HDMI_3D_TX_PHY_CKCALCTRL);
+
+ return 0;
+}
+
+static int hdmi_phy_configure(struct dw_hdmi *hdmi)
+{
+ const struct dw_hdmi_phy_data *phy = hdmi->phy.data;
+ const struct dw_hdmi_plat_data *pdata = hdmi->plat_data;
+ unsigned long mpixelclock = hdmi->hdmi_data.video_mode.mpixelclock;
+ int ret;
+
+ dw_hdmi_phy_power_off(hdmi);
/* Leave low power consumption mode by asserting SVSRET. */
- if (hdmi->phy->has_svsret)
+ if (phy->has_svsret)
dw_hdmi_phy_enable_svsret(hdmi, 1);
/* PHY reset. The reset signal is active high on Gen2 PHYs. */
HDMI_PHY_I2CM_SLAVE_ADDR);
hdmi_phy_test_clear(hdmi, 0);
- hdmi_phy_i2c_write(hdmi, mpll_config->res[0].cpce,
- HDMI_3D_TX_PHY_CPCE_CTRL);
- hdmi_phy_i2c_write(hdmi, mpll_config->res[0].gmp,
- HDMI_3D_TX_PHY_GMPCTRL);
- hdmi_phy_i2c_write(hdmi, curr_ctrl->curr[0],
- HDMI_3D_TX_PHY_CURRCTRL);
-
- hdmi_phy_i2c_write(hdmi, 0, HDMI_3D_TX_PHY_PLLPHBYCTRL);
- hdmi_phy_i2c_write(hdmi, HDMI_3D_TX_PHY_MSM_CTRL_CKO_SEL_FB_CLK,
- HDMI_3D_TX_PHY_MSM_CTRL);
-
- hdmi_phy_i2c_write(hdmi, phy_config->term, HDMI_3D_TX_PHY_TXTERM);
- hdmi_phy_i2c_write(hdmi, phy_config->sym_ctr,
- HDMI_3D_TX_PHY_CKSYMTXCTRL);
- hdmi_phy_i2c_write(hdmi, phy_config->vlev_ctr,
- HDMI_3D_TX_PHY_VLEVCTRL);
-
- /* Override and disable clock termination. */
- hdmi_phy_i2c_write(hdmi, HDMI_3D_TX_PHY_CKCALCTRL_OVERRIDE,
- HDMI_3D_TX_PHY_CKCALCTRL);
-
- dw_hdmi_phy_enable_powerdown(hdmi, false);
-
- /* toggle TMDS enable */
- dw_hdmi_phy_enable_tmds(hdmi, 0);
- dw_hdmi_phy_enable_tmds(hdmi, 1);
-
- /* gen2 tx power on */
- dw_hdmi_phy_gen2_txpwron(hdmi, 1);
- dw_hdmi_phy_gen2_pddq(hdmi, 0);
-
- /* Wait for PHY PLL lock */
- msec = 5;
- do {
- val = hdmi_readb(hdmi, HDMI_PHY_STAT0) & HDMI_PHY_TX_PHY_LOCK;
- if (!val)
- break;
-
- if (msec == 0) {
- dev_err(hdmi->dev, "PHY PLL not locked\n");
- return -ETIMEDOUT;
- }
-
- udelay(1000);
- msec--;
- } while (1);
+ /* Write to the PHY as configured by the platform */
+ if (pdata->configure_phy)
+ ret = pdata->configure_phy(hdmi, pdata, mpixelclock);
+ else
+ ret = phy->configure(hdmi, pdata, mpixelclock);
+ if (ret) {
+ dev_err(hdmi->dev, "PHY configuration failed (clock %lu)\n",
+ mpixelclock);
+ return ret;
+ }
- return 0;
+ return dw_hdmi_phy_power_on(hdmi);
}
-static int dw_hdmi_phy_init(struct dw_hdmi *hdmi)
+static int dw_hdmi_phy_init(struct dw_hdmi *hdmi, void *data,
+ struct drm_display_mode *mode)
{
int i, ret;
- bool cscon;
-
- /*check csc whether needed activated in HDMI mode */
- cscon = hdmi->sink_is_hdmi && is_color_space_conversion(hdmi);
/* HDMI Phy spec says to do the phy initialization sequence twice */
for (i = 0; i < 2; i++) {
dw_hdmi_phy_sel_data_en_pol(hdmi, 1);
dw_hdmi_phy_sel_interface_control(hdmi, 0);
- dw_hdmi_phy_enable_tmds(hdmi, 0);
- dw_hdmi_phy_enable_powerdown(hdmi, true);
- /* Enable CSC */
- ret = hdmi_phy_configure(hdmi, cscon);
+ ret = hdmi_phy_configure(hdmi);
if (ret)
return ret;
}
- hdmi->phy_enabled = true;
return 0;
}
+static void dw_hdmi_phy_disable(struct dw_hdmi *hdmi, void *data)
+{
+ dw_hdmi_phy_power_off(hdmi);
+}
+
+static enum drm_connector_status dw_hdmi_phy_read_hpd(struct dw_hdmi *hdmi,
+ void *data)
+{
+ return hdmi_readb(hdmi, HDMI_PHY_STAT0) & HDMI_PHY_HPD ?
+ connector_status_connected : connector_status_disconnected;
+}
+
+static const struct dw_hdmi_phy_ops dw_hdmi_synopsys_phy_ops = {
+ .init = dw_hdmi_phy_init,
+ .disable = dw_hdmi_phy_disable,
+ .read_hpd = dw_hdmi_phy_read_hpd,
+};
+
+/* -----------------------------------------------------------------------------
+ * HDMI TX Setup
+ */
+
static void hdmi_tx_hdcp_config(struct dw_hdmi *hdmi)
{
u8 de;
hdmi_writeb(hdmi, (frame.right_bar >> 8) & 0xff, HDMI_FC_AVISRB1);
}
+static void hdmi_config_vendor_specific_infoframe(struct dw_hdmi *hdmi,
+ struct drm_display_mode *mode)
+{
+ struct hdmi_vendor_infoframe frame;
+ u8 buffer[10];
+ ssize_t err;
+
+ err = drm_hdmi_vendor_infoframe_from_display_mode(&frame, mode);
+ if (err < 0)
+ /*
+ * Going into that statement does not means vendor infoframe
+ * fails. It just informed us that vendor infoframe is not
+ * needed for the selected mode. Only 4k or stereoscopic 3D
+ * mode requires vendor infoframe. So just simply return.
+ */
+ return;
+
+ err = hdmi_vendor_infoframe_pack(&frame, buffer, sizeof(buffer));
+ if (err < 0) {
+ dev_err(hdmi->dev, "Failed to pack vendor infoframe: %zd\n",
+ err);
+ return;
+ }
+ hdmi_mask_writeb(hdmi, 0, HDMI_FC_DATAUTO0, HDMI_FC_DATAUTO0_VSD_OFFSET,
+ HDMI_FC_DATAUTO0_VSD_MASK);
+
+ /* Set the length of HDMI vendor specific InfoFrame payload */
+ hdmi_writeb(hdmi, buffer[2], HDMI_FC_VSDSIZE);
+
+ /* Set 24bit IEEE Registration Identifier */
+ hdmi_writeb(hdmi, buffer[4], HDMI_FC_VSDIEEEID0);
+ hdmi_writeb(hdmi, buffer[5], HDMI_FC_VSDIEEEID1);
+ hdmi_writeb(hdmi, buffer[6], HDMI_FC_VSDIEEEID2);
+
+ /* Set HDMI_Video_Format and HDMI_VIC/3D_Structure */
+ hdmi_writeb(hdmi, buffer[7], HDMI_FC_VSDPAYLOAD0);
+ hdmi_writeb(hdmi, buffer[8], HDMI_FC_VSDPAYLOAD1);
+
+ if (frame.s3d_struct >= HDMI_3D_STRUCTURE_SIDE_BY_SIDE_HALF)
+ hdmi_writeb(hdmi, buffer[9], HDMI_FC_VSDPAYLOAD2);
+
+ /* Packet frame interpolation */
+ hdmi_writeb(hdmi, 1, HDMI_FC_DATAUTO1);
+
+ /* Auto packets per frame and line spacing */
+ hdmi_writeb(hdmi, 0x11, HDMI_FC_DATAUTO2);
+
+ /* Configures the Frame Composer On RDRB mode */
+ hdmi_mask_writeb(hdmi, 1, HDMI_FC_DATAUTO0, HDMI_FC_DATAUTO0_VSD_OFFSET,
+ HDMI_FC_DATAUTO0_VSD_MASK);
+}
+
static void hdmi_av_composer(struct dw_hdmi *hdmi,
const struct drm_display_mode *mode)
{
hdmi_writeb(hdmi, vsync_len, HDMI_FC_VSYNCINWIDTH);
}
-static void dw_hdmi_phy_disable(struct dw_hdmi *hdmi)
-{
- if (!hdmi->phy_enabled)
- return;
-
- dw_hdmi_phy_enable_tmds(hdmi, 0);
- dw_hdmi_phy_enable_powerdown(hdmi, true);
-
- hdmi->phy_enabled = false;
-}
-
/* HDMI Initialization Step B.4 */
static void dw_hdmi_enable_video_path(struct dw_hdmi *hdmi)
{
clkdis &= ~HDMI_MC_CLKDIS_CSCCLK_DISABLE;
hdmi_writeb(hdmi, clkdis, HDMI_MC_CLKDIS);
}
+
+ /* Enable color space conversion if needed */
+ if (is_color_space_conversion(hdmi))
+ hdmi_writeb(hdmi, HDMI_MC_FLOWCTRL_FEED_THROUGH_OFF_CSC_IN_PATH,
+ HDMI_MC_FLOWCTRL);
+ else
+ hdmi_writeb(hdmi, HDMI_MC_FLOWCTRL_FEED_THROUGH_OFF_CSC_BYPASS,
+ HDMI_MC_FLOWCTRL);
}
static void hdmi_enable_audio_clk(struct dw_hdmi *hdmi)
hdmi_av_composer(hdmi, mode);
/* HDMI Initializateion Step B.2 */
- ret = dw_hdmi_phy_init(hdmi);
+ ret = hdmi->phy.ops->init(hdmi, hdmi->phy.data, &hdmi->previous_mode);
if (ret)
return ret;
+ hdmi->phy.enabled = true;
/* HDMI Initialization Step B.3 */
dw_hdmi_enable_video_path(hdmi);
/* HDMI Initialization Step F - Configure AVI InfoFrame */
hdmi_config_AVI(hdmi, mode);
+ hdmi_config_vendor_specific_infoframe(hdmi, mode);
} else {
dev_dbg(hdmi->dev, "%s DVI mode\n", __func__);
}
static void dw_hdmi_poweroff(struct dw_hdmi *hdmi)
{
- dw_hdmi_phy_disable(hdmi);
+ if (hdmi->phy.enabled) {
+ hdmi->phy.ops->disable(hdmi, hdmi->phy.data);
+ hdmi->phy.enabled = false;
+ }
+
hdmi->bridge_is_on = false;
}
dw_hdmi_update_phy_mask(hdmi);
mutex_unlock(&hdmi->mutex);
- return hdmi_readb(hdmi, HDMI_PHY_STAT0) & HDMI_PHY_HPD ?
- connector_status_connected : connector_status_disconnected;
+ return hdmi->phy.ops->read_hpd(hdmi, hdmi->phy.data);
}
static int dw_hdmi_connector_get_modes(struct drm_connector *connector)
{
.type = DW_HDMI_PHY_DWC_HDMI_TX_PHY,
.name = "DWC HDMI TX PHY",
+ .gen = 1,
}, {
.type = DW_HDMI_PHY_DWC_MHL_PHY_HEAC,
.name = "DWC MHL PHY + HEAC PHY",
+ .gen = 2,
.has_svsret = true,
+ .configure = hdmi_phy_configure_dwc_hdmi_3d_tx,
}, {
.type = DW_HDMI_PHY_DWC_MHL_PHY,
.name = "DWC MHL PHY",
+ .gen = 2,
.has_svsret = true,
+ .configure = hdmi_phy_configure_dwc_hdmi_3d_tx,
}, {
.type = DW_HDMI_PHY_DWC_HDMI_3D_TX_PHY_HEAC,
.name = "DWC HDMI 3D TX PHY + HEAC PHY",
+ .gen = 2,
+ .configure = hdmi_phy_configure_dwc_hdmi_3d_tx,
}, {
.type = DW_HDMI_PHY_DWC_HDMI_3D_TX_PHY,
.name = "DWC HDMI 3D TX PHY",
+ .gen = 2,
+ .configure = hdmi_phy_configure_dwc_hdmi_3d_tx,
}, {
.type = DW_HDMI_PHY_DWC_HDMI20_TX_PHY,
.name = "DWC HDMI 2.0 TX PHY",
+ .gen = 2,
.has_svsret = true,
+ }, {
+ .type = DW_HDMI_PHY_VENDOR_PHY,
+ .name = "Vendor PHY",
}
};
phy_type = hdmi_readb(hdmi, HDMI_CONFIG2_ID);
+ if (phy_type == DW_HDMI_PHY_VENDOR_PHY) {
+ /* Vendor PHYs require support from the glue layer. */
+ if (!hdmi->plat_data->phy_ops || !hdmi->plat_data->phy_name) {
+ dev_err(hdmi->dev,
+ "Vendor HDMI PHY not supported by glue layer\n");
+ return -ENODEV;
+ }
+
+ hdmi->phy.ops = hdmi->plat_data->phy_ops;
+ hdmi->phy.data = hdmi->plat_data->phy_data;
+ hdmi->phy.name = hdmi->plat_data->phy_name;
+ return 0;
+ }
+
+ /* Synopsys PHYs are handled internally. */
for (i = 0; i < ARRAY_SIZE(dw_hdmi_phys); ++i) {
if (dw_hdmi_phys[i].type == phy_type) {
- hdmi->phy = &dw_hdmi_phys[i];
+ hdmi->phy.ops = &dw_hdmi_synopsys_phy_ops;
+ hdmi->phy.name = dw_hdmi_phys[i].name;
+ hdmi->phy.data = (void *)&dw_hdmi_phys[i];
+
+ if (!dw_hdmi_phys[i].configure &&
+ !hdmi->plat_data->configure_phy) {
+ dev_err(hdmi->dev, "%s requires platform support\n",
+ hdmi->phy.name);
+ return -ENODEV;
+ }
+
return 0;
}
}
- if (phy_type == DW_HDMI_PHY_VENDOR_PHY)
- dev_err(hdmi->dev, "Unsupported vendor HDMI PHY\n");
- else
- dev_err(hdmi->dev, "Unsupported HDMI PHY type (%02x)\n",
- phy_type);
-
+ dev_err(hdmi->dev, "Unsupported HDMI PHY type (%02x)\n", phy_type);
return -ENODEV;
}
+static const struct regmap_config hdmi_regmap_8bit_config = {
+ .reg_bits = 32,
+ .val_bits = 8,
+ .reg_stride = 1,
+ .max_register = HDMI_I2CM_FS_SCL_LCNT_0_ADDR,
+};
+
+static const struct regmap_config hdmi_regmap_32bit_config = {
+ .reg_bits = 32,
+ .val_bits = 32,
+ .reg_stride = 4,
+ .max_register = HDMI_I2CM_FS_SCL_LCNT_0_ADDR << 2,
+};
+
static struct dw_hdmi *
__dw_hdmi_probe(struct platform_device *pdev,
const struct dw_hdmi_plat_data *plat_data)
struct platform_device_info pdevinfo;
struct device_node *ddc_node;
struct dw_hdmi *hdmi;
- struct resource *iores;
+ struct resource *iores = NULL;
int irq;
int ret;
u32 val = 1;
hdmi->plat_data = plat_data;
hdmi->dev = dev;
- hdmi->dev_type = plat_data->dev_type;
hdmi->sample_rate = 48000;
hdmi->disabled = true;
hdmi->rxsense = true;
mutex_init(&hdmi->audio_mutex);
spin_lock_init(&hdmi->audio_lock);
- of_property_read_u32(np, "reg-io-width", &val);
-
- switch (val) {
- case 4:
- hdmi->write = dw_hdmi_writel;
- hdmi->read = dw_hdmi_readl;
- break;
- case 1:
- hdmi->write = dw_hdmi_writeb;
- hdmi->read = dw_hdmi_readb;
- break;
- default:
- dev_err(dev, "reg-io-width must be 1 or 4\n");
- return ERR_PTR(-EINVAL);
- }
-
ddc_node = of_parse_phandle(np, "ddc-i2c-bus", 0);
if (ddc_node) {
hdmi->ddc = of_get_i2c_adapter_by_node(ddc_node);
dev_dbg(hdmi->dev, "no ddc property found\n");
}
- iores = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- hdmi->regs = devm_ioremap_resource(dev, iores);
- if (IS_ERR(hdmi->regs)) {
- ret = PTR_ERR(hdmi->regs);
- goto err_res;
+ if (!plat_data->regm) {
+ const struct regmap_config *reg_config;
+
+ of_property_read_u32(np, "reg-io-width", &val);
+ switch (val) {
+ case 4:
+ reg_config = &hdmi_regmap_32bit_config;
+ hdmi->reg_shift = 2;
+ break;
+ case 1:
+ reg_config = &hdmi_regmap_8bit_config;
+ break;
+ default:
+ dev_err(dev, "reg-io-width must be 1 or 4\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ iores = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ hdmi->regs = devm_ioremap_resource(dev, iores);
+ if (IS_ERR(hdmi->regs)) {
+ ret = PTR_ERR(hdmi->regs);
+ goto err_res;
+ }
+
+ hdmi->regm = devm_regmap_init_mmio(dev, hdmi->regs, reg_config);
+ if (IS_ERR(hdmi->regm)) {
+ dev_err(dev, "Failed to configure regmap\n");
+ ret = PTR_ERR(hdmi->regm);
+ goto err_res;
+ }
+ } else {
+ hdmi->regm = plat_data->regm;
}
hdmi->isfr_clk = devm_clk_get(hdmi->dev, "isfr");
dev_info(dev, "Detected HDMI TX controller v%x.%03x %s HDCP (%s)\n",
hdmi->version >> 12, hdmi->version & 0xfff,
prod_id1 & HDMI_PRODUCT_ID1_HDCP ? "with" : "without",
- hdmi->phy->name);
+ hdmi->phy.name);
initialize_hdmi_ih_mutes(hdmi);
config0 = hdmi_readb(hdmi, HDMI_CONFIG0_ID);
config3 = hdmi_readb(hdmi, HDMI_CONFIG3_ID);
- if (config3 & HDMI_CONFIG3_AHBAUDDMA) {
+ if (iores && config3 & HDMI_CONFIG3_AHBAUDDMA) {
struct dw_hdmi_audio_data audio;
audio.phys = iores->start;
HDMI_FC_DBGFORCE_FORCEAUDIO = 0x10,
HDMI_FC_DBGFORCE_FORCEVIDEO = 0x1,
+/* FC_DATAUTO0 field values */
+ HDMI_FC_DATAUTO0_VSD_MASK = 0x08,
+ HDMI_FC_DATAUTO0_VSD_OFFSET = 3,
+
/* PHY_CONF0 field values */
HDMI_PHY_CONF0_PDZ_MASK = 0x80,
HDMI_PHY_CONF0_PDZ_OFFSET = 7,
};
MODULE_DEVICE_TABLE(of, tfp410_match);
-struct platform_driver tfp410_platform_driver = {
+static struct platform_driver tfp410_platform_driver = {
.probe = tfp410_probe,
.remove = tfp410_remove,
.driver = {
struct cirrus_framebuffer *gfb = &gfbdev->gfb;
drm_fb_helper_unregister_fbi(&gfbdev->helper);
- drm_fb_helper_release_fbi(&gfbdev->helper);
if (gfb->obj) {
drm_gem_object_unreference_unlocked(gfb->obj);
state->connectors[i].state);
state->connectors[i].ptr = NULL;
state->connectors[i].state = NULL;
- drm_connector_unreference(connector);
+ drm_connector_put(connector);
}
for (i = 0; i < config->num_crtc; i++) {
return ERR_PTR(-ENOMEM);
state->crtcs[index].state = crtc_state;
+ state->crtcs[index].old_state = crtc->state;
+ state->crtcs[index].new_state = crtc_state;
state->crtcs[index].ptr = crtc;
crtc_state->state = state;
if (mode && memcmp(&state->mode, mode, sizeof(*mode)) == 0)
return 0;
- drm_property_unreference_blob(state->mode_blob);
+ drm_property_blob_put(state->mode_blob);
state->mode_blob = NULL;
if (mode) {
if (blob == state->mode_blob)
return 0;
- drm_property_unreference_blob(state->mode_blob);
+ drm_property_blob_put(state->mode_blob);
state->mode_blob = NULL;
memset(&state->mode, 0, sizeof(state->mode));
blob->data))
return -EINVAL;
- state->mode_blob = drm_property_reference_blob(blob);
+ state->mode_blob = drm_property_blob_get(blob);
state->enable = true;
DRM_DEBUG_ATOMIC("Set [MODE:%s] for CRTC state %p\n",
state->mode.name, state);
if (old_blob == new_blob)
return;
- drm_property_unreference_blob(old_blob);
+ drm_property_blob_put(old_blob);
if (new_blob)
- drm_property_reference_blob(new_blob);
+ drm_property_blob_get(new_blob);
*blob = new_blob;
*replaced = true;
return -EINVAL;
if (expected_size > 0 && expected_size != new_blob->length) {
- drm_property_unreference_blob(new_blob);
+ drm_property_blob_put(new_blob);
return -EINVAL;
}
}
drm_atomic_replace_property_blob(blob, new_blob, replaced);
- drm_property_unreference_blob(new_blob);
+ drm_property_blob_put(new_blob);
return 0;
}
struct drm_property_blob *mode =
drm_property_lookup_blob(dev, val);
ret = drm_atomic_set_mode_prop_for_crtc(state, mode);
- drm_property_unreference_blob(mode);
+ drm_property_blob_put(mode);
return ret;
} else if (property == config->degamma_lut_property) {
ret = drm_atomic_replace_property_blob_from_id(crtc,
* pipe.
*/
if (state->event && !state->active && !crtc->state->active) {
- DRM_DEBUG_ATOMIC("[CRTC:%d] requesting event but off\n",
- crtc->base.id);
+ DRM_DEBUG_ATOMIC("[CRTC:%d:%s] requesting event but off\n",
+ crtc->base.id, crtc->name);
return -EINVAL;
}
state->planes[index].state = plane_state;
state->planes[index].ptr = plane;
+ state->planes[index].old_state = plane->state;
+ state->planes[index].new_state = plane_state;
plane_state->state = state;
DRM_DEBUG_ATOMIC("Added [PLANE:%d:%s] %p state to %p\n",
struct drm_framebuffer *fb = drm_framebuffer_lookup(dev, val);
drm_atomic_set_fb_for_plane(state, fb);
if (fb)
- drm_framebuffer_unreference(fb);
+ drm_framebuffer_put(fb);
} else if (property == config->prop_in_fence_fd) {
if (state->fence)
return -EINVAL;
if (!connector_state)
return ERR_PTR(-ENOMEM);
- drm_connector_reference(connector);
+ drm_connector_get(connector);
state->connectors[index].state = connector_state;
+ state->connectors[index].old_state = connector->state;
+ state->connectors[index].new_state = connector_state;
state->connectors[index].ptr = connector;
connector_state->state = state;
- DRM_DEBUG_ATOMIC("Added [CONNECTOR:%d] %p state to %p\n",
- connector->base.id, connector_state, state);
+ DRM_DEBUG_ATOMIC("Added [CONNECTOR:%d:%s] %p state to %p\n",
+ connector->base.id, connector->name,
+ connector_state, state);
if (connector_state->crtc) {
struct drm_crtc_state *crtc_state;
state->tv.saturation = val;
} else if (property == config->tv_hue_property) {
state->tv.hue = val;
+ } else if (property == config->link_status_property) {
+ /* Never downgrade from GOOD to BAD on userspace's request here,
+ * only hw issues can do that.
+ *
+ * For an atomic property the userspace doesn't need to be able
+ * to understand all the properties, but needs to be able to
+ * restore the state it wants on VT switch. So if the userspace
+ * tries to change the link_status from GOOD to BAD, driver
+ * silently rejects it and returns a 0. This prevents userspace
+ * from accidently breaking the display when it restores the
+ * state.
+ */
+ if (state->link_status != DRM_LINK_STATUS_GOOD)
+ state->link_status = val;
} else if (connector->funcs->atomic_set_property) {
return connector->funcs->atomic_set_property(connector,
state, property, val);
*val = state->tv.saturation;
} else if (property == config->tv_hue_property) {
*val = state->tv.hue;
+ } else if (property == config->link_status_property) {
+ *val = state->link_status;
} else if (connector->funcs->atomic_get_property) {
return connector->funcs->atomic_get_property(connector,
state, property, val);
return 0;
if (conn_state->crtc) {
- crtc_state = drm_atomic_get_existing_crtc_state(conn_state->state,
- conn_state->crtc);
+ crtc_state = drm_atomic_get_new_crtc_state(conn_state->state,
+ conn_state->crtc);
crtc_state->connector_mask &=
~(1 << drm_connector_index(conn_state->connector));
- drm_connector_unreference(conn_state->connector);
+ drm_connector_put(conn_state->connector);
conn_state->crtc = NULL;
}
crtc_state->connector_mask |=
1 << drm_connector_index(conn_state->connector);
- drm_connector_reference(conn_state->connector);
+ drm_connector_get(conn_state->connector);
conn_state->crtc = crtc;
DRM_DEBUG_ATOMIC("Link connector state %p to [CRTC:%d:%s]\n",
struct drm_connector *connector;
struct drm_connector_state *conn_state;
struct drm_connector_list_iter conn_iter;
+ struct drm_crtc_state *crtc_state;
int ret;
+ crtc_state = drm_atomic_get_crtc_state(state, crtc);
+ if (IS_ERR(crtc_state))
+ return PTR_ERR(crtc_state);
+
ret = drm_modeset_lock(&config->connection_mutex, state->acquire_ctx);
if (ret)
return ret;
crtc->base.id, crtc->name, state);
/*
- * Changed connectors are already in @state, so only need to look at the
- * current configuration.
+ * Changed connectors are already in @state, so only need to look
+ * at the connector_mask in crtc_state.
*/
- drm_connector_list_iter_get(state->dev, &conn_iter);
+ drm_connector_list_iter_begin(state->dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
- if (connector->state->crtc != crtc)
+ if (!(crtc_state->connector_mask & (1 << drm_connector_index(connector))))
continue;
conn_state = drm_atomic_get_connector_state(state, connector);
if (IS_ERR(conn_state)) {
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
return PTR_ERR(conn_state);
}
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
return 0;
}
{
struct drm_plane *plane;
- WARN_ON(!drm_atomic_get_existing_crtc_state(state, crtc));
+ WARN_ON(!drm_atomic_get_new_crtc_state(state, crtc));
drm_for_each_plane_mask(plane, state->dev, crtc->state->plane_mask) {
struct drm_plane_state *plane_state =
DRM_DEBUG_ATOMIC("checking %p\n", state);
- for_each_plane_in_state(state, plane, plane_state, i) {
+ for_each_new_plane_in_state(state, plane, plane_state, i) {
ret = drm_atomic_plane_check(plane, plane_state);
if (ret) {
DRM_DEBUG_ATOMIC("[PLANE:%d:%s] atomic core check failed\n",
}
}
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
ret = drm_atomic_crtc_check(crtc, crtc_state);
if (ret) {
DRM_DEBUG_ATOMIC("[CRTC:%d:%s] atomic core check failed\n",
ret = config->funcs->atomic_check(state->dev, state);
if (!state->allow_modeset) {
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
if (drm_atomic_crtc_needs_modeset(crtc_state)) {
DRM_DEBUG_ATOMIC("[CRTC:%d:%s] requires full modeset\n",
crtc->base.id, crtc->name);
DRM_DEBUG_ATOMIC("checking %p\n", state);
- for_each_plane_in_state(state, plane, plane_state, i)
+ for_each_new_plane_in_state(state, plane, plane_state, i)
drm_atomic_plane_print_state(&p, plane_state);
- for_each_crtc_in_state(state, crtc, crtc_state, i)
+ for_each_new_crtc_in_state(state, crtc, crtc_state, i)
drm_atomic_crtc_print_state(&p, crtc_state);
- for_each_connector_in_state(state, connector, connector_state, i)
+ for_each_new_connector_in_state(state, connector, connector_state, i)
drm_atomic_connector_print_state(&p, connector_state);
}
list_for_each_entry(crtc, &config->crtc_list, head)
drm_atomic_crtc_print_state(p, crtc->state);
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter)
drm_atomic_connector_print_state(p, connector->state);
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
}
EXPORT_SYMBOL(drm_state_dump);
if (ret == 0) {
struct drm_framebuffer *new_fb = plane->state->fb;
if (new_fb)
- drm_framebuffer_reference(new_fb);
+ drm_framebuffer_get(new_fb);
plane->fb = new_fb;
plane->crtc = plane->state->crtc;
if (plane->old_fb)
- drm_framebuffer_unreference(plane->old_fb);
+ drm_framebuffer_put(plane->old_fb);
}
plane->old_fb = NULL;
}
if (arg->flags & DRM_MODE_ATOMIC_TEST_ONLY)
return 0;
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
s32 __user *fence_ptr;
fence_ptr = get_out_fence_for_crtc(crtc_state->state, crtc);
return;
}
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
struct drm_pending_vblank_event *event = crtc_state->event;
/*
* Free the allocated event. drm_atomic_helper_setup_commit
kfree(fence_state);
}
+int drm_atomic_remove_fb(struct drm_framebuffer *fb)
+{
+ struct drm_modeset_acquire_ctx ctx;
+ struct drm_device *dev = fb->dev;
+ struct drm_atomic_state *state;
+ struct drm_plane *plane;
+ struct drm_connector *conn;
+ struct drm_connector_state *conn_state;
+ int i, ret = 0;
+ unsigned plane_mask;
+
+ state = drm_atomic_state_alloc(dev);
+ if (!state)
+ return -ENOMEM;
+
+ drm_modeset_acquire_init(&ctx, 0);
+ state->acquire_ctx = &ctx;
+
+retry:
+ plane_mask = 0;
+ ret = drm_modeset_lock_all_ctx(dev, &ctx);
+ if (ret)
+ goto unlock;
+
+ drm_for_each_plane(plane, dev) {
+ struct drm_plane_state *plane_state;
+
+ if (plane->state->fb != fb)
+ continue;
+
+ plane_state = drm_atomic_get_plane_state(state, plane);
+ if (IS_ERR(plane_state)) {
+ ret = PTR_ERR(plane_state);
+ goto unlock;
+ }
+
+ if (plane_state->crtc->primary == plane) {
+ struct drm_crtc_state *crtc_state;
+
+ crtc_state = drm_atomic_get_existing_crtc_state(state, plane_state->crtc);
+
+ ret = drm_atomic_add_affected_connectors(state, plane_state->crtc);
+ if (ret)
+ goto unlock;
+
+ crtc_state->active = false;
+ ret = drm_atomic_set_mode_for_crtc(crtc_state, NULL);
+ if (ret)
+ goto unlock;
+ }
+
+ drm_atomic_set_fb_for_plane(plane_state, NULL);
+ ret = drm_atomic_set_crtc_for_plane(plane_state, NULL);
+ if (ret)
+ goto unlock;
+
+ plane_mask |= BIT(drm_plane_index(plane));
+
+ plane->old_fb = plane->fb;
+ }
+
+ for_each_connector_in_state(state, conn, conn_state, i) {
+ ret = drm_atomic_set_crtc_for_connector(conn_state, NULL);
+
+ if (ret)
+ goto unlock;
+ }
+
+ if (plane_mask)
+ ret = drm_atomic_commit(state);
+
+unlock:
+ if (plane_mask)
+ drm_atomic_clean_old_fb(dev, plane_mask, ret);
+
+ if (ret == -EDEADLK) {
+ drm_modeset_backoff(&ctx);
+ goto retry;
+ }
+
+ drm_atomic_state_put(state);
+
+ drm_modeset_drop_locks(&ctx);
+ drm_modeset_acquire_fini(&ctx);
+
+ return ret;
+}
+
int drm_mode_atomic_ioctl(struct drm_device *dev,
void *data, struct drm_file *file_priv)
{
}
if (!obj->properties) {
- drm_mode_object_unreference(obj);
+ drm_mode_object_put(obj);
ret = -ENOENT;
goto out;
}
if (get_user(count_props, count_props_ptr + copied_objs)) {
- drm_mode_object_unreference(obj);
+ drm_mode_object_put(obj);
ret = -EFAULT;
goto out;
}
struct drm_property *prop;
if (get_user(prop_id, props_ptr + copied_props)) {
- drm_mode_object_unreference(obj);
+ drm_mode_object_put(obj);
ret = -EFAULT;
goto out;
}
prop = drm_mode_obj_find_prop_id(obj, prop_id);
if (!prop) {
- drm_mode_object_unreference(obj);
+ drm_mode_object_put(obj);
ret = -ENOENT;
goto out;
}
if (copy_from_user(&prop_value,
prop_values_ptr + copied_props,
sizeof(prop_value))) {
- drm_mode_object_unreference(obj);
+ drm_mode_object_put(obj);
ret = -EFAULT;
goto out;
}
ret = atomic_set_prop(state, obj, prop, prop_value);
if (ret) {
- drm_mode_object_unreference(obj);
+ drm_mode_object_put(obj);
goto out;
}
plane_mask |= (1 << drm_plane_index(plane));
plane->old_fb = plane->fb;
}
- drm_mode_object_unreference(obj);
+ drm_mode_object_put(obj);
}
ret = prepare_crtc_signaling(dev, state, arg, file_priv, &fence_state,
*/
static void
drm_atomic_helper_plane_changed(struct drm_atomic_state *state,
+ struct drm_plane_state *old_plane_state,
struct drm_plane_state *plane_state,
struct drm_plane *plane)
{
struct drm_crtc_state *crtc_state;
- if (plane->state->crtc) {
- crtc_state = drm_atomic_get_existing_crtc_state(state,
- plane->state->crtc);
+ if (old_plane_state->crtc) {
+ crtc_state = drm_atomic_get_new_crtc_state(state,
+ old_plane_state->crtc);
if (WARN_ON(!crtc_state))
return;
}
if (plane_state->crtc) {
- crtc_state = drm_atomic_get_existing_crtc_state(state,
- plane_state->crtc);
+ crtc_state = drm_atomic_get_new_crtc_state(state, plane_state->crtc);
if (WARN_ON(!crtc_state))
return;
static int handle_conflicting_encoders(struct drm_atomic_state *state,
bool disable_conflicting_encoders)
{
- struct drm_connector_state *conn_state;
+ struct drm_connector_state *new_conn_state;
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
struct drm_encoder *encoder;
* part of the state. If the same encoder is assigned to multiple
* connectors bail out.
*/
- for_each_connector_in_state(state, connector, conn_state, i) {
+ for_each_new_connector_in_state(state, connector, new_conn_state, i) {
const struct drm_connector_helper_funcs *funcs = connector->helper_private;
struct drm_encoder *new_encoder;
- if (!conn_state->crtc)
+ if (!new_conn_state->crtc)
continue;
if (funcs->atomic_best_encoder)
- new_encoder = funcs->atomic_best_encoder(connector, conn_state);
+ new_encoder = funcs->atomic_best_encoder(connector, new_conn_state);
else if (funcs->best_encoder)
new_encoder = funcs->best_encoder(connector);
else
* and the crtc is disabled if no encoder is left. This preserves
* compatibility with the legacy set_config behavior.
*/
- drm_connector_list_iter_get(state->dev, &conn_iter);
+ drm_connector_list_iter_begin(state->dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
struct drm_crtc_state *crtc_state;
- if (drm_atomic_get_existing_connector_state(state, connector))
+ if (drm_atomic_get_new_connector_state(state, connector))
continue;
encoder = connector->state->best_encoder;
goto out;
}
- conn_state = drm_atomic_get_connector_state(state, connector);
- if (IS_ERR(conn_state)) {
- ret = PTR_ERR(conn_state);
+ new_conn_state = drm_atomic_get_connector_state(state, connector);
+ if (IS_ERR(new_conn_state)) {
+ ret = PTR_ERR(new_conn_state);
goto out;
}
DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] in use on [CRTC:%d:%s], disabling [CONNECTOR:%d:%s]\n",
encoder->base.id, encoder->name,
- conn_state->crtc->base.id, conn_state->crtc->name,
+ new_conn_state->crtc->base.id, new_conn_state->crtc->name,
connector->base.id, connector->name);
- crtc_state = drm_atomic_get_existing_crtc_state(state, conn_state->crtc);
+ crtc_state = drm_atomic_get_new_crtc_state(state, new_conn_state->crtc);
- ret = drm_atomic_set_crtc_for_connector(conn_state, NULL);
+ ret = drm_atomic_set_crtc_for_connector(new_conn_state, NULL);
if (ret)
goto out;
}
}
out:
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
return ret;
}
*/
WARN_ON(!crtc && encoder != conn_state->best_encoder);
if (crtc) {
- crtc_state = drm_atomic_get_existing_crtc_state(state, crtc);
+ crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
crtc_state->encoder_mask &=
~(1 << drm_encoder_index(conn_state->best_encoder));
crtc = conn_state->crtc;
WARN_ON(!crtc);
if (crtc) {
- crtc_state = drm_atomic_get_existing_crtc_state(state, crtc);
+ crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
crtc_state->encoder_mask |=
1 << drm_encoder_index(encoder);
{
struct drm_crtc_state *crtc_state;
struct drm_connector *connector;
- struct drm_connector_state *connector_state;
+ struct drm_connector_state *old_connector_state, *new_connector_state;
int i;
- for_each_connector_in_state(state, connector, connector_state, i) {
+ for_each_oldnew_connector_in_state(state, connector, old_connector_state, new_connector_state, i) {
struct drm_crtc *encoder_crtc;
- if (connector_state->best_encoder != encoder)
+ if (new_connector_state->best_encoder != encoder)
continue;
- encoder_crtc = connector->state->crtc;
+ encoder_crtc = old_connector_state->crtc;
DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] in use on [CRTC:%d:%s], stealing it\n",
encoder->base.id, encoder->name,
encoder_crtc->base.id, encoder_crtc->name);
- set_best_encoder(state, connector_state, NULL);
+ set_best_encoder(state, new_connector_state, NULL);
- crtc_state = drm_atomic_get_existing_crtc_state(state, encoder_crtc);
+ crtc_state = drm_atomic_get_new_crtc_state(state, encoder_crtc);
crtc_state->connectors_changed = true;
return;
static int
update_connector_routing(struct drm_atomic_state *state,
struct drm_connector *connector,
- struct drm_connector_state *connector_state)
+ struct drm_connector_state *old_connector_state,
+ struct drm_connector_state *new_connector_state)
{
const struct drm_connector_helper_funcs *funcs;
struct drm_encoder *new_encoder;
connector->base.id,
connector->name);
- if (connector->state->crtc != connector_state->crtc) {
- if (connector->state->crtc) {
- crtc_state = drm_atomic_get_existing_crtc_state(state, connector->state->crtc);
+ if (old_connector_state->crtc != new_connector_state->crtc) {
+ if (old_connector_state->crtc) {
+ crtc_state = drm_atomic_get_new_crtc_state(state, old_connector_state->crtc);
crtc_state->connectors_changed = true;
}
- if (connector_state->crtc) {
- crtc_state = drm_atomic_get_existing_crtc_state(state, connector_state->crtc);
+ if (new_connector_state->crtc) {
+ crtc_state = drm_atomic_get_new_crtc_state(state, new_connector_state->crtc);
crtc_state->connectors_changed = true;
}
}
- if (!connector_state->crtc) {
+ if (!new_connector_state->crtc) {
DRM_DEBUG_ATOMIC("Disabling [CONNECTOR:%d:%s]\n",
connector->base.id,
connector->name);
- set_best_encoder(state, connector_state, NULL);
+ set_best_encoder(state, new_connector_state, NULL);
return 0;
}
if (funcs->atomic_best_encoder)
new_encoder = funcs->atomic_best_encoder(connector,
- connector_state);
+ new_connector_state);
else if (funcs->best_encoder)
new_encoder = funcs->best_encoder(connector);
else
return -EINVAL;
}
- if (!drm_encoder_crtc_ok(new_encoder, connector_state->crtc)) {
- DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] incompatible with [CRTC:%d]\n",
+ if (!drm_encoder_crtc_ok(new_encoder, new_connector_state->crtc)) {
+ DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] incompatible with [CRTC:%d:%s]\n",
new_encoder->base.id,
new_encoder->name,
- connector_state->crtc->base.id);
+ new_connector_state->crtc->base.id,
+ new_connector_state->crtc->name);
return -EINVAL;
}
- if (new_encoder == connector_state->best_encoder) {
- set_best_encoder(state, connector_state, new_encoder);
+ if (new_encoder == new_connector_state->best_encoder) {
+ set_best_encoder(state, new_connector_state, new_encoder);
DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] keeps [ENCODER:%d:%s], now on [CRTC:%d:%s]\n",
connector->base.id,
connector->name,
new_encoder->base.id,
new_encoder->name,
- connector_state->crtc->base.id,
- connector_state->crtc->name);
+ new_connector_state->crtc->base.id,
+ new_connector_state->crtc->name);
return 0;
}
steal_encoder(state, new_encoder);
- set_best_encoder(state, connector_state, new_encoder);
+ set_best_encoder(state, new_connector_state, new_encoder);
- crtc_state = drm_atomic_get_existing_crtc_state(state, connector_state->crtc);
+ crtc_state = drm_atomic_get_new_crtc_state(state, new_connector_state->crtc);
crtc_state->connectors_changed = true;
DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] using [ENCODER:%d:%s] on [CRTC:%d:%s]\n",
connector->name,
new_encoder->base.id,
new_encoder->name,
- connector_state->crtc->base.id,
- connector_state->crtc->name);
+ new_connector_state->crtc->base.id,
+ new_connector_state->crtc->name);
return 0;
}
mode_fixup(struct drm_atomic_state *state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *new_crtc_state;
struct drm_connector *connector;
- struct drm_connector_state *conn_state;
+ struct drm_connector_state *new_conn_state;
int i;
int ret;
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
- if (!crtc_state->mode_changed &&
- !crtc_state->connectors_changed)
+ for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
+ if (!new_crtc_state->mode_changed &&
+ !new_crtc_state->connectors_changed)
continue;
- drm_mode_copy(&crtc_state->adjusted_mode, &crtc_state->mode);
+ drm_mode_copy(&new_crtc_state->adjusted_mode, &new_crtc_state->mode);
}
- for_each_connector_in_state(state, connector, conn_state, i) {
+ for_each_new_connector_in_state(state, connector, new_conn_state, i) {
const struct drm_encoder_helper_funcs *funcs;
struct drm_encoder *encoder;
- WARN_ON(!!conn_state->best_encoder != !!conn_state->crtc);
+ WARN_ON(!!new_conn_state->best_encoder != !!new_conn_state->crtc);
- if (!conn_state->crtc || !conn_state->best_encoder)
+ if (!new_conn_state->crtc || !new_conn_state->best_encoder)
continue;
- crtc_state = drm_atomic_get_existing_crtc_state(state,
- conn_state->crtc);
+ new_crtc_state =
+ drm_atomic_get_new_crtc_state(state, new_conn_state->crtc);
/*
* Each encoder has at most one connector (since we always steal
* it away), so we won't call ->mode_fixup twice.
*/
- encoder = conn_state->best_encoder;
+ encoder = new_conn_state->best_encoder;
funcs = encoder->helper_private;
- ret = drm_bridge_mode_fixup(encoder->bridge, &crtc_state->mode,
- &crtc_state->adjusted_mode);
+ ret = drm_bridge_mode_fixup(encoder->bridge, &new_crtc_state->mode,
+ &new_crtc_state->adjusted_mode);
if (!ret) {
DRM_DEBUG_ATOMIC("Bridge fixup failed\n");
return -EINVAL;
}
if (funcs && funcs->atomic_check) {
- ret = funcs->atomic_check(encoder, crtc_state,
- conn_state);
+ ret = funcs->atomic_check(encoder, new_crtc_state,
+ new_conn_state);
if (ret) {
DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] check failed\n",
encoder->base.id, encoder->name);
return ret;
}
} else if (funcs && funcs->mode_fixup) {
- ret = funcs->mode_fixup(encoder, &crtc_state->mode,
- &crtc_state->adjusted_mode);
+ ret = funcs->mode_fixup(encoder, &new_crtc_state->mode,
+ &new_crtc_state->adjusted_mode);
if (!ret) {
DRM_DEBUG_ATOMIC("[ENCODER:%d:%s] fixup failed\n",
encoder->base.id, encoder->name);
}
}
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
const struct drm_crtc_helper_funcs *funcs;
- if (!crtc_state->enable)
+ if (!new_crtc_state->enable)
continue;
- if (!crtc_state->mode_changed &&
- !crtc_state->connectors_changed)
+ if (!new_crtc_state->mode_changed &&
+ !new_crtc_state->connectors_changed)
continue;
funcs = crtc->helper_private;
if (!funcs->mode_fixup)
continue;
- ret = funcs->mode_fixup(crtc, &crtc_state->mode,
- &crtc_state->adjusted_mode);
+ ret = funcs->mode_fixup(crtc, &new_crtc_state->mode,
+ &new_crtc_state->adjusted_mode);
if (!ret) {
DRM_DEBUG_ATOMIC("[CRTC:%d:%s] fixup failed\n",
crtc->base.id, crtc->name);
struct drm_atomic_state *state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
struct drm_connector *connector;
- struct drm_connector_state *connector_state;
+ struct drm_connector_state *old_connector_state, *new_connector_state;
int i, ret;
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
- if (!drm_mode_equal(&crtc->state->mode, &crtc_state->mode)) {
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
+ if (!drm_mode_equal(&old_crtc_state->mode, &new_crtc_state->mode)) {
DRM_DEBUG_ATOMIC("[CRTC:%d:%s] mode changed\n",
crtc->base.id, crtc->name);
- crtc_state->mode_changed = true;
+ new_crtc_state->mode_changed = true;
}
- if (crtc->state->enable != crtc_state->enable) {
+ if (old_crtc_state->enable != new_crtc_state->enable) {
DRM_DEBUG_ATOMIC("[CRTC:%d:%s] enable changed\n",
crtc->base.id, crtc->name);
* The other way around is true as well. enable != 0
* iff connectors are attached and a mode is set.
*/
- crtc_state->mode_changed = true;
- crtc_state->connectors_changed = true;
+ new_crtc_state->mode_changed = true;
+ new_crtc_state->connectors_changed = true;
}
}
if (ret)
return ret;
- for_each_connector_in_state(state, connector, connector_state, i) {
+ for_each_oldnew_connector_in_state(state, connector, old_connector_state, new_connector_state, i) {
/*
* This only sets crtc->connectors_changed for routing changes,
* drivers must set crtc->connectors_changed themselves when
* connector properties need to be updated.
*/
ret = update_connector_routing(state, connector,
- connector_state);
+ old_connector_state,
+ new_connector_state);
if (ret)
return ret;
+ if (old_connector_state->crtc) {
+ new_crtc_state = drm_atomic_get_new_crtc_state(state,
+ old_connector_state->crtc);
+ if (old_connector_state->link_status !=
+ new_connector_state->link_status)
+ new_crtc_state->connectors_changed = true;
+ }
}
/*
* configuration. This must be done before calling mode_fixup in case a
* crtc only changed its mode but has the same set of connectors.
*/
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
bool has_connectors =
- !!crtc_state->connector_mask;
+ !!new_crtc_state->connector_mask;
/*
* We must set ->active_changed after walking connectors for
* otherwise an update that only changes active would result in
* a full modeset because update_connector_routing force that.
*/
- if (crtc->state->active != crtc_state->active) {
+ if (old_crtc_state->active != new_crtc_state->active) {
DRM_DEBUG_ATOMIC("[CRTC:%d:%s] active changed\n",
crtc->base.id, crtc->name);
- crtc_state->active_changed = true;
+ new_crtc_state->active_changed = true;
}
- if (!drm_atomic_crtc_needs_modeset(crtc_state))
+ if (!drm_atomic_crtc_needs_modeset(new_crtc_state))
continue;
DRM_DEBUG_ATOMIC("[CRTC:%d:%s] needs all connectors, enable: %c, active: %c\n",
crtc->base.id, crtc->name,
- crtc_state->enable ? 'y' : 'n',
- crtc_state->active ? 'y' : 'n');
+ new_crtc_state->enable ? 'y' : 'n',
+ new_crtc_state->active ? 'y' : 'n');
ret = drm_atomic_add_affected_connectors(state, crtc);
if (ret != 0)
if (ret != 0)
return ret;
- if (crtc_state->enable != has_connectors) {
+ if (new_crtc_state->enable != has_connectors) {
DRM_DEBUG_ATOMIC("[CRTC:%d:%s] enabled/connectors mismatch\n",
crtc->base.id, crtc->name);
struct drm_atomic_state *state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *new_crtc_state;
struct drm_plane *plane;
- struct drm_plane_state *plane_state;
+ struct drm_plane_state *new_plane_state, *old_plane_state;
int i, ret = 0;
- for_each_plane_in_state(state, plane, plane_state, i) {
+ for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) {
const struct drm_plane_helper_funcs *funcs;
funcs = plane->helper_private;
- drm_atomic_helper_plane_changed(state, plane_state, plane);
+ drm_atomic_helper_plane_changed(state, old_plane_state, new_plane_state, plane);
if (!funcs || !funcs->atomic_check)
continue;
- ret = funcs->atomic_check(plane, plane_state);
+ ret = funcs->atomic_check(plane, new_plane_state);
if (ret) {
DRM_DEBUG_ATOMIC("[PLANE:%d:%s] atomic driver check failed\n",
plane->base.id, plane->name);
}
}
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
const struct drm_crtc_helper_funcs *funcs;
funcs = crtc->helper_private;
if (!funcs || !funcs->atomic_check)
continue;
- ret = funcs->atomic_check(crtc, crtc_state);
+ ret = funcs->atomic_check(crtc, new_crtc_state);
if (ret) {
DRM_DEBUG_ATOMIC("[CRTC:%d:%s] atomic driver check failed\n",
crtc->base.id, crtc->name);
disable_outputs(struct drm_device *dev, struct drm_atomic_state *old_state)
{
struct drm_connector *connector;
- struct drm_connector_state *old_conn_state;
+ struct drm_connector_state *old_conn_state, *new_conn_state;
struct drm_crtc *crtc;
- struct drm_crtc_state *old_crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
int i;
- for_each_connector_in_state(old_state, connector, old_conn_state, i) {
+ for_each_oldnew_connector_in_state(old_state, connector, old_conn_state, new_conn_state, i) {
const struct drm_encoder_helper_funcs *funcs;
struct drm_encoder *encoder;
if (!old_conn_state->crtc)
continue;
- old_crtc_state = drm_atomic_get_existing_crtc_state(old_state,
- old_conn_state->crtc);
+ old_crtc_state = drm_atomic_get_old_crtc_state(old_state, old_conn_state->crtc);
if (!old_crtc_state->active ||
!drm_atomic_crtc_needs_modeset(old_conn_state->crtc->state))
/* Right function depends upon target state. */
if (funcs) {
- if (connector->state->crtc && funcs->prepare)
+ if (new_conn_state->crtc && funcs->prepare)
funcs->prepare(encoder);
else if (funcs->disable)
funcs->disable(encoder);
drm_bridge_post_disable(encoder->bridge);
}
- for_each_crtc_in_state(old_state, crtc, old_crtc_state, i) {
+ for_each_oldnew_crtc_in_state(old_state, crtc, old_crtc_state, new_crtc_state, i) {
const struct drm_crtc_helper_funcs *funcs;
/* Shut down everything that needs a full modeset. */
- if (!drm_atomic_crtc_needs_modeset(crtc->state))
+ if (!drm_atomic_crtc_needs_modeset(new_crtc_state))
continue;
if (!old_crtc_state->active)
/* Right function depends upon target state. */
- if (crtc->state->enable && funcs->prepare)
+ if (new_crtc_state->enable && funcs->prepare)
funcs->prepare(crtc);
else if (funcs->atomic_disable)
funcs->atomic_disable(crtc, old_crtc_state);
struct drm_atomic_state *old_state)
{
struct drm_connector *connector;
- struct drm_connector_state *old_conn_state;
+ struct drm_connector_state *old_conn_state, *new_conn_state;
struct drm_crtc *crtc;
- struct drm_crtc_state *old_crtc_state;
+ struct drm_crtc_state *new_crtc_state;
int i;
/* clear out existing links and update dpms */
- for_each_connector_in_state(old_state, connector, old_conn_state, i) {
+ for_each_oldnew_connector_in_state(old_state, connector, old_conn_state, new_conn_state, i) {
if (connector->encoder) {
WARN_ON(!connector->encoder->crtc);
connector->encoder = NULL;
}
- crtc = connector->state->crtc;
+ crtc = new_conn_state->crtc;
if ((!crtc && old_conn_state->crtc) ||
(crtc && drm_atomic_crtc_needs_modeset(crtc->state))) {
struct drm_property *dpms_prop =
}
/* set new links */
- for_each_connector_in_state(old_state, connector, old_conn_state, i) {
- if (!connector->state->crtc)
+ for_each_new_connector_in_state(old_state, connector, new_conn_state, i) {
+ if (!new_conn_state->crtc)
continue;
- if (WARN_ON(!connector->state->best_encoder))
+ if (WARN_ON(!new_conn_state->best_encoder))
continue;
- connector->encoder = connector->state->best_encoder;
- connector->encoder->crtc = connector->state->crtc;
+ connector->encoder = new_conn_state->best_encoder;
+ connector->encoder->crtc = new_conn_state->crtc;
}
/* set legacy state in the crtc structure */
- for_each_crtc_in_state(old_state, crtc, old_crtc_state, i) {
+ for_each_new_crtc_in_state(old_state, crtc, new_crtc_state, i) {
struct drm_plane *primary = crtc->primary;
+ struct drm_plane_state *new_plane_state;
+
+ crtc->mode = new_crtc_state->mode;
+ crtc->enabled = new_crtc_state->enable;
- crtc->mode = crtc->state->mode;
- crtc->enabled = crtc->state->enable;
+ new_plane_state =
+ drm_atomic_get_new_plane_state(old_state, primary);
- if (drm_atomic_get_existing_plane_state(old_state, primary) &&
- primary->state->crtc == crtc) {
- crtc->x = primary->state->src_x >> 16;
- crtc->y = primary->state->src_y >> 16;
+ if (new_plane_state && new_plane_state->crtc == crtc) {
+ crtc->x = new_plane_state->src_x >> 16;
+ crtc->y = new_plane_state->src_y >> 16;
}
- if (crtc->state->enable)
+ if (new_crtc_state->enable)
drm_calc_timestamping_constants(crtc,
- &crtc->state->adjusted_mode);
+ &new_crtc_state->adjusted_mode);
}
}
EXPORT_SYMBOL(drm_atomic_helper_update_legacy_modeset_state);
crtc_set_mode(struct drm_device *dev, struct drm_atomic_state *old_state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *old_crtc_state;
+ struct drm_crtc_state *new_crtc_state;
struct drm_connector *connector;
- struct drm_connector_state *old_conn_state;
+ struct drm_connector_state *new_conn_state;
int i;
- for_each_crtc_in_state(old_state, crtc, old_crtc_state, i) {
+ for_each_new_crtc_in_state(old_state, crtc, new_crtc_state, i) {
const struct drm_crtc_helper_funcs *funcs;
- if (!crtc->state->mode_changed)
+ if (!new_crtc_state->mode_changed)
continue;
funcs = crtc->helper_private;
- if (crtc->state->enable && funcs->mode_set_nofb) {
+ if (new_crtc_state->enable && funcs->mode_set_nofb) {
DRM_DEBUG_ATOMIC("modeset on [CRTC:%d:%s]\n",
crtc->base.id, crtc->name);
}
}
- for_each_connector_in_state(old_state, connector, old_conn_state, i) {
+ for_each_new_connector_in_state(old_state, connector, new_conn_state, i) {
const struct drm_encoder_helper_funcs *funcs;
- struct drm_crtc_state *new_crtc_state;
struct drm_encoder *encoder;
struct drm_display_mode *mode, *adjusted_mode;
- if (!connector->state->best_encoder)
+ if (!new_conn_state->best_encoder)
continue;
- encoder = connector->state->best_encoder;
+ encoder = new_conn_state->best_encoder;
funcs = encoder->helper_private;
- new_crtc_state = connector->state->crtc->state;
+ new_crtc_state = new_conn_state->crtc->state;
mode = &new_crtc_state->mode;
adjusted_mode = &new_crtc_state->adjusted_mode;
*/
if (funcs && funcs->atomic_mode_set) {
funcs->atomic_mode_set(encoder, new_crtc_state,
- connector->state);
+ new_conn_state);
} else if (funcs && funcs->mode_set) {
funcs->mode_set(encoder, mode, adjusted_mode);
}
struct drm_atomic_state *old_state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *old_crtc_state;
+ struct drm_crtc_state *new_crtc_state;
struct drm_connector *connector;
- struct drm_connector_state *old_conn_state;
+ struct drm_connector_state *new_conn_state;
int i;
- for_each_crtc_in_state(old_state, crtc, old_crtc_state, i) {
+ for_each_new_crtc_in_state(old_state, crtc, new_crtc_state, i) {
const struct drm_crtc_helper_funcs *funcs;
/* Need to filter out CRTCs where only planes change. */
- if (!drm_atomic_crtc_needs_modeset(crtc->state))
+ if (!drm_atomic_crtc_needs_modeset(new_crtc_state))
continue;
- if (!crtc->state->active)
+ if (!new_crtc_state->active)
continue;
funcs = crtc->helper_private;
- if (crtc->state->enable) {
+ if (new_crtc_state->enable) {
DRM_DEBUG_ATOMIC("enabling [CRTC:%d:%s]\n",
crtc->base.id, crtc->name);
}
}
- for_each_connector_in_state(old_state, connector, old_conn_state, i) {
+ for_each_new_connector_in_state(old_state, connector, new_conn_state, i) {
const struct drm_encoder_helper_funcs *funcs;
struct drm_encoder *encoder;
- if (!connector->state->best_encoder)
+ if (!new_conn_state->best_encoder)
continue;
- if (!connector->state->crtc->state->active ||
- !drm_atomic_crtc_needs_modeset(connector->state->crtc->state))
+ if (!new_conn_state->crtc->state->active ||
+ !drm_atomic_crtc_needs_modeset(new_conn_state->crtc->state))
continue;
- encoder = connector->state->best_encoder;
+ encoder = new_conn_state->best_encoder;
funcs = encoder->helper_private;
DRM_DEBUG_ATOMIC("enabling [ENCODER:%d:%s]\n",
bool pre_swap)
{
struct drm_plane *plane;
- struct drm_plane_state *plane_state;
+ struct drm_plane_state *new_plane_state;
int i, ret;
- for_each_plane_in_state(state, plane, plane_state, i) {
- if (!pre_swap)
- plane_state = plane->state;
-
- if (!plane_state->fence)
+ for_each_new_plane_in_state(state, plane, new_plane_state, i) {
+ if (!new_plane_state->fence)
continue;
- WARN_ON(!plane_state->fb);
+ WARN_ON(!new_plane_state->fb);
/*
* If waiting for fences pre-swap (ie: nonblock), userspace can
* still interrupt the operation. Instead of blocking until the
* timer expires, make the wait interruptible.
*/
- ret = dma_fence_wait(plane_state->fence, pre_swap);
+ ret = dma_fence_wait(new_plane_state->fence, pre_swap);
if (ret)
return ret;
- dma_fence_put(plane_state->fence);
- plane_state->fence = NULL;
+ dma_fence_put(new_plane_state->fence);
+ new_plane_state->fence = NULL;
}
return 0;
struct drm_atomic_state *old_state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *old_crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
int i, ret;
unsigned crtc_mask = 0;
if (old_state->legacy_cursor_update)
return;
- for_each_crtc_in_state(old_state, crtc, old_crtc_state, i) {
- struct drm_crtc_state *new_crtc_state = crtc->state;
-
+ for_each_oldnew_crtc_in_state(old_state, crtc, old_crtc_state, new_crtc_state, i) {
if (!new_crtc_state->active || !new_crtc_state->planes_changed)
continue;
old_state->crtcs[i].last_vblank_count = drm_crtc_vblank_count(crtc);
}
- for_each_crtc_in_state(old_state, crtc, old_crtc_state, i) {
+ for_each_old_crtc_in_state(old_state, crtc, old_crtc_state, i) {
if (!(crtc_mask & drm_crtc_mask(crtc)))
continue;
drm_crtc_vblank_count(crtc),
msecs_to_jiffies(50));
- WARN(!ret, "[CRTC:%d] vblank wait timed out\n", crtc->base.id);
+ WARN(!ret, "[CRTC:%d:%s] vblank wait timed out\n",
+ crtc->base.id, crtc->name);
drm_crtc_vblank_put(crtc);
}
static void commit_tail(struct drm_atomic_state *old_state)
{
struct drm_device *dev = old_state->dev;
- struct drm_mode_config_helper_funcs *funcs;
+ const struct drm_mode_config_helper_funcs *funcs;
funcs = dev->mode_config.helper_private;
bool nonblock)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
struct drm_crtc_commit *commit;
int i, ret;
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
commit = kzalloc(sizeof(*commit), GFP_KERNEL);
if (!commit)
return -ENOMEM;
/* Drivers only send out events when at least either current or
* new CRTC state is active. Complete right away if everything
* stays off. */
- if (!crtc->state->active && !crtc_state->active) {
+ if (!old_crtc_state->active && !new_crtc_state->active) {
complete_all(&commit->flip_done);
continue;
}
continue;
}
- if (!crtc_state->event) {
+ if (!new_crtc_state->event) {
commit->event = kzalloc(sizeof(*commit->event),
GFP_KERNEL);
if (!commit->event)
return -ENOMEM;
- crtc_state->event = commit->event;
+ new_crtc_state->event = commit->event;
}
- crtc_state->event->base.completion = &commit->flip_done;
- crtc_state->event->base.completion_release = release_crtc_commit;
+ new_crtc_state->event->base.completion = &commit->flip_done;
+ new_crtc_state->event->base.completion_release = release_crtc_commit;
drm_crtc_commit_get(commit);
}
void drm_atomic_helper_wait_for_dependencies(struct drm_atomic_state *old_state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *new_crtc_state;
struct drm_crtc_commit *commit;
int i;
long ret;
- for_each_crtc_in_state(old_state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(old_state, crtc, new_crtc_state, i) {
spin_lock(&crtc->commit_lock);
commit = preceeding_commit(crtc);
if (commit)
void drm_atomic_helper_commit_hw_done(struct drm_atomic_state *old_state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *new_crtc_state;
struct drm_crtc_commit *commit;
int i;
- for_each_crtc_in_state(old_state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(old_state, crtc, new_crtc_state, i) {
commit = old_state->crtcs[i].commit;
if (!commit)
continue;
/* backend must have consumed any event by now */
- WARN_ON(crtc->state->event);
+ WARN_ON(new_crtc_state->event);
spin_lock(&crtc->commit_lock);
complete_all(&commit->hw_done);
spin_unlock(&crtc->commit_lock);
void drm_atomic_helper_commit_cleanup_done(struct drm_atomic_state *old_state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *new_crtc_state;
struct drm_crtc_commit *commit;
int i;
long ret;
- for_each_crtc_in_state(old_state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(old_state, crtc, new_crtc_state, i) {
commit = old_state->crtcs[i].commit;
if (WARN_ON(!commit))
continue;
struct drm_atomic_state *state)
{
struct drm_plane *plane;
- struct drm_plane_state *plane_state;
+ struct drm_plane_state *new_plane_state;
int ret, i, j;
- for_each_plane_in_state(state, plane, plane_state, i) {
+ for_each_new_plane_in_state(state, plane, new_plane_state, i) {
const struct drm_plane_helper_funcs *funcs;
funcs = plane->helper_private;
if (funcs->prepare_fb) {
- ret = funcs->prepare_fb(plane, plane_state);
+ ret = funcs->prepare_fb(plane, new_plane_state);
if (ret)
goto fail;
}
return 0;
fail:
- for_each_plane_in_state(state, plane, plane_state, j) {
+ for_each_new_plane_in_state(state, plane, new_plane_state, j) {
const struct drm_plane_helper_funcs *funcs;
if (j >= i)
funcs = plane->helper_private;
if (funcs->cleanup_fb)
- funcs->cleanup_fb(plane, plane_state);
+ funcs->cleanup_fb(plane, new_plane_state);
}
return ret;
uint32_t flags)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *old_crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
struct drm_plane *plane;
- struct drm_plane_state *old_plane_state;
+ struct drm_plane_state *old_plane_state, *new_plane_state;
int i;
bool active_only = flags & DRM_PLANE_COMMIT_ACTIVE_ONLY;
bool no_disable = flags & DRM_PLANE_COMMIT_NO_DISABLE_AFTER_MODESET;
- for_each_crtc_in_state(old_state, crtc, old_crtc_state, i) {
+ for_each_oldnew_crtc_in_state(old_state, crtc, old_crtc_state, new_crtc_state, i) {
const struct drm_crtc_helper_funcs *funcs;
funcs = crtc->helper_private;
if (!funcs || !funcs->atomic_begin)
continue;
- if (active_only && !crtc->state->active)
+ if (active_only && !new_crtc_state->active)
continue;
funcs->atomic_begin(crtc, old_crtc_state);
}
- for_each_plane_in_state(old_state, plane, old_plane_state, i) {
+ for_each_oldnew_plane_in_state(old_state, plane, old_plane_state, new_plane_state, i) {
const struct drm_plane_helper_funcs *funcs;
bool disabling;
if (!funcs)
continue;
- disabling = drm_atomic_plane_disabling(plane, old_plane_state);
+ disabling = drm_atomic_plane_disabling(old_plane_state,
+ new_plane_state);
if (active_only) {
/*
* CRTC to avoid skipping planes being disabled on an
* active CRTC.
*/
- if (!disabling && !plane_crtc_active(plane->state))
+ if (!disabling && !plane_crtc_active(new_plane_state))
continue;
if (disabling && !plane_crtc_active(old_plane_state))
continue;
continue;
funcs->atomic_disable(plane, old_plane_state);
- } else if (plane->state->crtc || disabling) {
+ } else if (new_plane_state->crtc || disabling) {
funcs->atomic_update(plane, old_plane_state);
}
}
- for_each_crtc_in_state(old_state, crtc, old_crtc_state, i) {
+ for_each_oldnew_crtc_in_state(old_state, crtc, old_crtc_state, new_crtc_state, i) {
const struct drm_crtc_helper_funcs *funcs;
funcs = crtc->helper_private;
if (!funcs || !funcs->atomic_flush)
continue;
- if (active_only && !crtc->state->active)
+ if (active_only && !new_crtc_state->active)
continue;
funcs->atomic_flush(crtc, old_crtc_state);
drm_for_each_plane_mask(plane, crtc->dev, plane_mask) {
struct drm_plane_state *old_plane_state =
- drm_atomic_get_existing_plane_state(old_state, plane);
+ drm_atomic_get_old_plane_state(old_state, plane);
const struct drm_plane_helper_funcs *plane_funcs;
plane_funcs = plane->helper_private;
WARN_ON(plane->state->crtc && plane->state->crtc != crtc);
- if (drm_atomic_plane_disabling(plane, old_plane_state) &&
+ if (drm_atomic_plane_disabling(old_plane_state, plane->state) &&
plane_funcs->atomic_disable)
plane_funcs->atomic_disable(plane, old_plane_state);
else if (plane->state->crtc ||
- drm_atomic_plane_disabling(plane, old_plane_state))
+ drm_atomic_plane_disabling(old_plane_state, plane->state))
plane_funcs->atomic_update(plane, old_plane_state);
}
struct drm_atomic_state *old_state)
{
struct drm_plane *plane;
- struct drm_plane_state *plane_state;
+ struct drm_plane_state *old_plane_state, *new_plane_state;
int i;
- for_each_plane_in_state(old_state, plane, plane_state, i) {
+ for_each_oldnew_plane_in_state(old_state, plane, old_plane_state, new_plane_state, i) {
const struct drm_plane_helper_funcs *funcs;
+ struct drm_plane_state *plane_state;
+
+ /*
+ * This might be called before swapping when commit is aborted,
+ * in which case we have to cleanup the new state.
+ */
+ if (old_plane_state == plane->state)
+ plane_state = new_plane_state;
+ else
+ plane_state = old_plane_state;
funcs = plane->helper_private;
int i;
long ret;
struct drm_connector *connector;
- struct drm_connector_state *conn_state;
+ struct drm_connector_state *old_conn_state, *new_conn_state;
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
struct drm_plane *plane;
- struct drm_plane_state *plane_state;
+ struct drm_plane_state *old_plane_state, *new_plane_state;
struct drm_crtc_commit *commit;
if (stall) {
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
spin_lock(&crtc->commit_lock);
commit = list_first_entry_or_null(&crtc->commit_list,
struct drm_crtc_commit, commit_entry);
}
}
- for_each_connector_in_state(state, connector, conn_state, i) {
- connector->state->state = state;
- swap(state->connectors[i].state, connector->state);
- connector->state->state = NULL;
+ for_each_oldnew_connector_in_state(state, connector, old_conn_state, new_conn_state, i) {
+ WARN_ON(connector->state != old_conn_state);
+
+ old_conn_state->state = state;
+ new_conn_state->state = NULL;
+
+ state->connectors[i].state = old_conn_state;
+ connector->state = new_conn_state;
}
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
- crtc->state->state = state;
- swap(state->crtcs[i].state, crtc->state);
- crtc->state->state = NULL;
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
+ WARN_ON(crtc->state != old_crtc_state);
+
+ old_crtc_state->state = state;
+ new_crtc_state->state = NULL;
+
+ state->crtcs[i].state = old_crtc_state;
+ crtc->state = new_crtc_state;
if (state->crtcs[i].commit) {
spin_lock(&crtc->commit_lock);
}
}
- for_each_plane_in_state(state, plane, plane_state, i) {
- plane->state->state = state;
- swap(state->planes[i].state, plane->state);
- plane->state->state = NULL;
+ for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) {
+ WARN_ON(plane->state != old_plane_state);
+
+ old_plane_state->state = state;
+ new_plane_state->state = NULL;
+
+ state->planes[i].state = old_plane_state;
+ plane->state = new_plane_state;
}
}
EXPORT_SYMBOL(drm_atomic_helper_swap_state);
{
struct drm_device *dev = set->crtc->dev;
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *new_crtc_state;
struct drm_connector *connector;
- struct drm_connector_state *conn_state;
+ struct drm_connector_state *new_conn_state;
int ret, i;
ret = drm_modeset_lock(&dev->mode_config.connection_mutex,
if (ret)
return ret;
- for_each_connector_in_state(state, connector, conn_state, i) {
- if (conn_state->crtc == set->crtc) {
- ret = drm_atomic_set_crtc_for_connector(conn_state,
+ for_each_new_connector_in_state(state, connector, new_conn_state, i) {
+ if (new_conn_state->crtc == set->crtc) {
+ ret = drm_atomic_set_crtc_for_connector(new_conn_state,
NULL);
if (ret)
return ret;
+
+ /* Make sure legacy setCrtc always re-trains */
+ new_conn_state->link_status = DRM_LINK_STATUS_GOOD;
}
}
/* Then set all connectors from set->connectors on the target crtc */
for (i = 0; i < set->num_connectors; i++) {
- conn_state = drm_atomic_get_connector_state(state,
+ new_conn_state = drm_atomic_get_connector_state(state,
set->connectors[i]);
- if (IS_ERR(conn_state))
- return PTR_ERR(conn_state);
+ if (IS_ERR(new_conn_state))
+ return PTR_ERR(new_conn_state);
- ret = drm_atomic_set_crtc_for_connector(conn_state,
+ ret = drm_atomic_set_crtc_for_connector(new_conn_state,
set->crtc);
if (ret)
return ret;
}
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
/* Don't update ->enable for the CRTC in the set_config request,
* since a mismatch would indicate a bug in the upper layers.
* The actual modeset code later on will catch any
if (crtc == set->crtc)
continue;
- if (!crtc_state->connector_mask) {
- ret = drm_atomic_set_mode_prop_for_crtc(crtc_state,
+ if (!new_crtc_state->connector_mask) {
+ ret = drm_atomic_set_mode_prop_for_crtc(new_crtc_state,
NULL);
if (ret < 0)
return ret;
- crtc_state->active = false;
+ new_crtc_state->active = false;
}
}
*
* Provides a default crtc set_config handler using the atomic driver interface.
*
+ * NOTE: For backwards compatibility with old userspace this automatically
+ * resets the "link-status" property to GOOD, to force any link
+ * re-training. The SETCRTC ioctl does not define whether an update does
+ * need a full modeset or just a plane update, hence we're allowed to do
+ * that. See also drm_mode_connector_set_link_status_property().
+ *
* Returns:
* Returns 0 on success, negative errno numbers on failure.
*/
struct drm_modeset_acquire_ctx *ctx)
{
struct drm_atomic_state *state;
+ struct drm_connector_state *conn_state;
struct drm_connector *conn;
- struct drm_connector_list_iter conn_iter;
- int err;
+ struct drm_plane_state *plane_state;
+ struct drm_plane *plane;
+ struct drm_crtc_state *crtc_state;
+ struct drm_crtc *crtc;
+ int ret, i;
state = drm_atomic_state_alloc(dev);
if (!state)
state->acquire_ctx = ctx;
- drm_connector_list_iter_get(dev, &conn_iter);
- drm_for_each_connector_iter(conn, &conn_iter) {
- struct drm_crtc *crtc = conn->state->crtc;
- struct drm_crtc_state *crtc_state;
-
- if (!crtc || conn->dpms != DRM_MODE_DPMS_ON)
- continue;
-
+ drm_for_each_crtc(crtc, dev) {
crtc_state = drm_atomic_get_crtc_state(state, crtc);
if (IS_ERR(crtc_state)) {
- err = PTR_ERR(crtc_state);
+ ret = PTR_ERR(crtc_state);
goto free;
}
crtc_state->active = false;
+
+ ret = drm_atomic_set_mode_prop_for_crtc(crtc_state, NULL);
+ if (ret < 0)
+ goto free;
+
+ ret = drm_atomic_add_affected_planes(state, crtc);
+ if (ret < 0)
+ goto free;
+
+ ret = drm_atomic_add_affected_connectors(state, crtc);
+ if (ret < 0)
+ goto free;
+ }
+
+ for_each_connector_in_state(state, conn, conn_state, i) {
+ ret = drm_atomic_set_crtc_for_connector(conn_state, NULL);
+ if (ret < 0)
+ goto free;
+ }
+
+ for_each_plane_in_state(state, plane, plane_state, i) {
+ ret = drm_atomic_set_crtc_for_plane(plane_state, NULL);
+ if (ret < 0)
+ goto free;
+
+ drm_atomic_set_fb_for_plane(plane_state, NULL);
}
- err = drm_atomic_commit(state);
+ ret = drm_atomic_commit(state);
free:
- drm_connector_list_iter_put(&conn_iter);
drm_atomic_state_put(state);
- return err;
+ return ret;
}
+
EXPORT_SYMBOL(drm_atomic_helper_disable_all);
/**
*
* See also:
* drm_atomic_helper_duplicate_state(), drm_atomic_helper_disable_all(),
- * drm_atomic_helper_resume()
+ * drm_atomic_helper_resume(), drm_atomic_helper_commit_duplicated_state()
*/
struct drm_atomic_state *drm_atomic_helper_suspend(struct drm_device *dev)
{
}
EXPORT_SYMBOL(drm_atomic_helper_suspend);
+/**
+ * drm_atomic_helper_commit_duplicated_state - commit duplicated state
+ * @state: duplicated atomic state to commit
+ * @ctx: pointer to acquire_ctx to use for commit.
+ *
+ * The state returned by drm_atomic_helper_duplicate_state() and
+ * drm_atomic_helper_suspend() is partially invalid, and needs to
+ * be fixed up before commit.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ *
+ * See also:
+ * drm_atomic_helper_suspend()
+ */
+int drm_atomic_helper_commit_duplicated_state(struct drm_atomic_state *state,
+ struct drm_modeset_acquire_ctx *ctx)
+{
+ int i;
+ struct drm_plane *plane;
+ struct drm_plane_state *new_plane_state;
+ struct drm_connector *connector;
+ struct drm_connector_state *new_conn_state;
+ struct drm_crtc *crtc;
+ struct drm_crtc_state *new_crtc_state;
+
+ state->acquire_ctx = ctx;
+
+ for_each_new_plane_in_state(state, plane, new_plane_state, i)
+ state->planes[i].old_state = plane->state;
+
+ for_each_new_crtc_in_state(state, crtc, new_crtc_state, i)
+ state->crtcs[i].old_state = crtc->state;
+
+ for_each_new_connector_in_state(state, connector, new_conn_state, i)
+ state->connectors[i].old_state = connector->state;
+
+ return drm_atomic_commit(state);
+}
+EXPORT_SYMBOL(drm_atomic_helper_commit_duplicated_state);
+
/**
* drm_atomic_helper_resume - subsystem-level resume helper
* @dev: DRM device
int err;
drm_mode_config_reset(dev);
+
drm_modeset_lock_all(dev);
- state->acquire_ctx = config->acquire_ctx;
- err = drm_atomic_commit(state);
+ err = drm_atomic_helper_commit_duplicated_state(state, config->acquire_ctx);
drm_modeset_unlock_all(dev);
return err;
struct drm_atomic_state *state,
struct drm_crtc *crtc,
struct drm_framebuffer *fb,
- struct drm_pending_vblank_event *event)
+ struct drm_pending_vblank_event *event,
+ uint32_t flags)
{
struct drm_plane *plane = crtc->primary;
struct drm_plane_state *plane_state;
return PTR_ERR(crtc_state);
crtc_state->event = event;
+ crtc_state->pageflip_flags = flags;
plane_state = drm_atomic_get_plane_state(state, plane);
if (IS_ERR(plane_state))
return PTR_ERR(plane_state);
-
ret = drm_atomic_set_crtc_for_plane(plane_state, crtc);
if (ret != 0)
return ret;
/* Make sure we don't accidentally do a full modeset. */
state->allow_modeset = false;
if (!crtc_state->active) {
- DRM_DEBUG_ATOMIC("[CRTC:%d] disabled, rejecting legacy flip\n",
- crtc->base.id);
+ DRM_DEBUG_ATOMIC("[CRTC:%d:%s] disabled, rejecting legacy flip\n",
+ crtc->base.id, crtc->name);
return -EINVAL;
}
* Provides a default &drm_crtc_funcs.page_flip implementation
* using the atomic driver interface.
*
- * Note that for now so called async page flips (i.e. updates which are not
- * synchronized to vblank) are not supported, since the atomic interfaces have
- * no provisions for this yet.
- *
* Returns:
* Returns 0 on success, negative errno numbers on failure.
*
struct drm_atomic_state *state;
int ret = 0;
- if (flags & DRM_MODE_PAGE_FLIP_ASYNC)
- return -EINVAL;
-
state = drm_atomic_state_alloc(plane->dev);
if (!state)
return -ENOMEM;
state->acquire_ctx = drm_modeset_legacy_acquire_ctx(crtc);
retry:
- ret = page_flip_common(state, crtc, fb, event);
+ ret = page_flip_common(state, crtc, fb, event, flags);
if (ret != 0)
goto fail;
struct drm_crtc_state *crtc_state;
int ret = 0;
- if (flags & DRM_MODE_PAGE_FLIP_ASYNC)
- return -EINVAL;
-
state = drm_atomic_state_alloc(plane->dev);
if (!state)
return -ENOMEM;
state->acquire_ctx = drm_modeset_legacy_acquire_ctx(crtc);
retry:
- ret = page_flip_common(state, crtc, fb, event);
+ ret = page_flip_common(state, crtc, fb, event, flags);
if (ret != 0)
goto fail;
- crtc_state = drm_atomic_get_existing_crtc_state(state, crtc);
+ crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
if (WARN_ON(!crtc_state)) {
ret = -EINVAL;
goto fail;
WARN_ON(!drm_modeset_is_locked(&config->connection_mutex));
- drm_connector_list_iter_get(connector->dev, &conn_iter);
+ drm_connector_list_iter_begin(connector->dev, &conn_iter);
drm_for_each_connector_iter(tmp_connector, &conn_iter) {
if (tmp_connector->state->crtc != crtc)
continue;
break;
}
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
crtc_state->active = active;
ret = drm_atomic_commit(state);
memcpy(state, crtc->state, sizeof(*state));
if (state->mode_blob)
- drm_property_reference_blob(state->mode_blob);
+ drm_property_blob_get(state->mode_blob);
if (state->degamma_lut)
- drm_property_reference_blob(state->degamma_lut);
+ drm_property_blob_get(state->degamma_lut);
if (state->ctm)
- drm_property_reference_blob(state->ctm);
+ drm_property_blob_get(state->ctm);
if (state->gamma_lut)
- drm_property_reference_blob(state->gamma_lut);
+ drm_property_blob_get(state->gamma_lut);
state->mode_changed = false;
state->active_changed = false;
state->planes_changed = false;
state->color_mgmt_changed = false;
state->zpos_changed = false;
state->event = NULL;
+ state->pageflip_flags = 0;
}
EXPORT_SYMBOL(__drm_atomic_helper_crtc_duplicate_state);
*/
void __drm_atomic_helper_crtc_destroy_state(struct drm_crtc_state *state)
{
- drm_property_unreference_blob(state->mode_blob);
- drm_property_unreference_blob(state->degamma_lut);
- drm_property_unreference_blob(state->ctm);
- drm_property_unreference_blob(state->gamma_lut);
+ drm_property_blob_put(state->mode_blob);
+ drm_property_blob_put(state->degamma_lut);
+ drm_property_blob_put(state->ctm);
+ drm_property_blob_put(state->gamma_lut);
}
EXPORT_SYMBOL(__drm_atomic_helper_crtc_destroy_state);
memcpy(state, plane->state, sizeof(*state));
if (state->fb)
- drm_framebuffer_reference(state->fb);
+ drm_framebuffer_get(state->fb);
state->fence = NULL;
}
void __drm_atomic_helper_plane_destroy_state(struct drm_plane_state *state)
{
if (state->fb)
- drm_framebuffer_unreference(state->fb);
+ drm_framebuffer_put(state->fb);
if (state->fence)
dma_fence_put(state->fence);
{
memcpy(state, connector->state, sizeof(*state));
if (state->crtc)
- drm_connector_reference(connector);
+ drm_connector_get(connector);
}
EXPORT_SYMBOL(__drm_atomic_helper_connector_duplicate_state);
}
}
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(conn, &conn_iter) {
struct drm_connector_state *conn_state;
conn_state = drm_atomic_get_connector_state(state, conn);
if (IS_ERR(conn_state)) {
err = PTR_ERR(conn_state);
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
goto free;
}
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
/* clear the acquire context so that it isn't accidentally reused */
state->acquire_ctx = NULL;
__drm_atomic_helper_connector_destroy_state(struct drm_connector_state *state)
{
if (state->crtc)
- drm_connector_unreference(state->connector);
+ drm_connector_put(state->connector);
}
EXPORT_SYMBOL(__drm_atomic_helper_connector_destroy_state);
goto backoff;
drm_atomic_state_put(state);
- drm_property_unreference_blob(blob);
+ drm_property_blob_put(blob);
return ret;
backoff:
struct drm_atomic_state *state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
struct drm_plane *plane;
- struct drm_plane_state *plane_state;
+ struct drm_plane_state *old_plane_state, *new_plane_state;
int i, ret = 0;
- for_each_plane_in_state(state, plane, plane_state, i) {
- crtc = plane_state->crtc;
+ for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) {
+ crtc = new_plane_state->crtc;
if (!crtc)
continue;
- if (plane->state->zpos != plane_state->zpos) {
- crtc_state =
- drm_atomic_get_existing_crtc_state(state, crtc);
- crtc_state->zpos_changed = true;
+ if (old_plane_state->zpos != new_plane_state->zpos) {
+ new_crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
+ new_crtc_state->zpos_changed = true;
}
}
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
- if (crtc_state->plane_mask != crtc->state->plane_mask ||
- crtc_state->zpos_changed) {
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
+ if (old_crtc_state->plane_mask != new_crtc_state->plane_mask ||
+ new_crtc_state->zpos_changed) {
ret = drm_atomic_helper_crtc_normalize_zpos(crtc,
- crtc_state);
+ new_crtc_state);
if (ret)
return ret;
}
}
if (wbinvd_on_all_cpus())
- printk(KERN_ERR "Timed out waiting for cache flush.\n");
+ pr_err("Timed out waiting for cache flush\n");
#elif defined(__powerpc__)
unsigned long i;
kunmap_atomic(page_virtual);
}
#else
- printk(KERN_ERR "Architecture has no drm_cache.c support\n");
+ pr_err("Architecture has no drm_cache.c support\n");
WARN_ON_ONCE(1);
#endif
}
}
if (wbinvd_on_all_cpus())
- printk(KERN_ERR "Timed out waiting for cache flush.\n");
+ pr_err("Timed out waiting for cache flush\n");
#else
- printk(KERN_ERR "Architecture has no drm_cache.c support\n");
+ pr_err("Architecture has no drm_cache.c support\n");
WARN_ON_ONCE(1);
#endif
}
}
if (wbinvd_on_all_cpus())
- printk(KERN_ERR "Timed out waiting for cache flush.\n");
+ pr_err("Timed out waiting for cache flush\n");
#else
- printk(KERN_ERR "Architecture has no drm_cache.c support\n");
+ pr_err("Architecture has no drm_cache.c support\n");
WARN_ON_ONCE(1);
#endif
}
* als fixed panels or anything else that can display pixels in some form. As
* opposed to all other KMS objects representing hardware (like CRTC, encoder or
* plane abstractions) connectors can be hotplugged and unplugged at runtime.
- * Hence they are reference-counted using drm_connector_reference() and
- * drm_connector_unreference().
+ * Hence they are reference-counted using drm_connector_get() and
+ * drm_connector_put().
*
* KMS driver must create, initialize, register and attach at a &struct
* drm_connector for each such sink. The instance is created as other KMS
return;
if (mode->force) {
- const char *s;
-
- switch (mode->force) {
- case DRM_FORCE_OFF:
- s = "OFF";
- break;
- case DRM_FORCE_ON_DIGITAL:
- s = "ON - dig";
- break;
- default:
- case DRM_FORCE_ON:
- s = "ON";
- break;
- }
-
- DRM_INFO("forcing %s connector %s\n", connector->name, s);
+ DRM_INFO("forcing %s connector %s\n", connector->name,
+ drm_get_connector_force_name(mode->force));
connector->force = mode->force;
}
struct ida *connector_ida =
&drm_connector_enum_list[connector_type].ida;
- ret = drm_mode_object_get_reg(dev, &connector->base,
- DRM_MODE_OBJECT_CONNECTOR,
- false, drm_connector_free);
+ ret = __drm_mode_object_add(dev, &connector->base,
+ DRM_MODE_OBJECT_CONNECTOR,
+ false, drm_connector_free);
if (ret)
return ret;
drm_object_attach_property(&connector->base,
config->dpms_property, 0);
+ drm_object_attach_property(&connector->base,
+ config->link_status_property,
+ 0);
+
if (drm_core_check_feature(dev, DRIVER_ATOMIC)) {
drm_object_attach_property(&connector->base, config->prop_crtc_id, 0);
}
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter)
drm_connector_unregister(connector);
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
}
int drm_connector_register_all(struct drm_device *dev)
struct drm_connector_list_iter conn_iter;
int ret = 0;
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
ret = drm_connector_register(connector);
if (ret)
break;
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
if (ret)
drm_connector_unregister_all(dev);
}
EXPORT_SYMBOL(drm_get_connector_status_name);
+/**
+ * drm_get_connector_force_name - return a string for connector force
+ * @force: connector force to get name of
+ *
+ * Returns: const pointer to name.
+ */
+const char *drm_get_connector_force_name(enum drm_connector_force force)
+{
+ switch (force) {
+ case DRM_FORCE_UNSPECIFIED:
+ return "unspecified";
+ case DRM_FORCE_OFF:
+ return "off";
+ case DRM_FORCE_ON:
+ return "on";
+ case DRM_FORCE_ON_DIGITAL:
+ return "digital";
+ default:
+ return "unknown";
+ }
+}
+
#ifdef CONFIG_LOCKDEP
static struct lockdep_map connector_list_iter_dep_map = {
.name = "drm_connector_list_iter"
#endif
/**
- * drm_connector_list_iter_get - initialize a connector_list iterator
+ * drm_connector_list_iter_begin - initialize a connector_list iterator
* @dev: DRM device
* @iter: connector_list iterator
*
* Sets @iter up to walk the &drm_mode_config.connector_list of @dev. @iter
- * must always be cleaned up again by calling drm_connector_list_iter_put().
+ * must always be cleaned up again by calling drm_connector_list_iter_end().
* Iteration itself happens using drm_connector_list_iter_next() or
* drm_for_each_connector_iter().
*/
-void drm_connector_list_iter_get(struct drm_device *dev,
- struct drm_connector_list_iter *iter)
+void drm_connector_list_iter_begin(struct drm_device *dev,
+ struct drm_connector_list_iter *iter)
{
iter->dev = dev;
iter->conn = NULL;
lock_acquire_shared_recursive(&connector_list_iter_dep_map, 0, 1, NULL, _RET_IP_);
}
-EXPORT_SYMBOL(drm_connector_list_iter_get);
+EXPORT_SYMBOL(drm_connector_list_iter_begin);
/**
* drm_connector_list_iter_next - return next connector
spin_unlock_irqrestore(&config->connector_list_lock, flags);
if (old_conn)
- drm_connector_unreference(old_conn);
+ drm_connector_put(old_conn);
return iter->conn;
}
EXPORT_SYMBOL(drm_connector_list_iter_next);
/**
- * drm_connector_list_iter_put - tear down a connector_list iterator
+ * drm_connector_list_iter_end - tear down a connector_list iterator
* @iter: connector_list iterator
*
* Tears down @iter and releases any resources (like &drm_connector references)
* iteration completes fully or when it was aborted without walking the entire
* list.
*/
-void drm_connector_list_iter_put(struct drm_connector_list_iter *iter)
+void drm_connector_list_iter_end(struct drm_connector_list_iter *iter)
{
iter->dev = NULL;
if (iter->conn)
- drm_connector_unreference(iter->conn);
+ drm_connector_put(iter->conn);
lock_release(&connector_list_iter_dep_map, 0, _RET_IP_);
}
-EXPORT_SYMBOL(drm_connector_list_iter_put);
+EXPORT_SYMBOL(drm_connector_list_iter_end);
static const struct drm_prop_enum_list drm_subpixel_enum_list[] = {
{ SubPixelUnknown, "Unknown" },
};
DRM_ENUM_NAME_FN(drm_get_dpms_name, drm_dpms_enum_list)
+static const struct drm_prop_enum_list drm_link_status_enum_list[] = {
+ { DRM_MODE_LINK_STATUS_GOOD, "Good" },
+ { DRM_MODE_LINK_STATUS_BAD, "Bad" },
+};
+DRM_ENUM_NAME_FN(drm_get_link_status_name, drm_link_status_enum_list)
+
/**
* drm_display_info_set_bus_formats - set the supported bus formats
* @info: display info to store bus formats in
* tiling and virtualize both &drm_crtc and &drm_plane if needed. Drivers
* should update this value using drm_mode_connector_set_tile_property().
* Userspace cannot change this property.
+ * link-status:
+ * Connector link-status property to indicate the status of link. The default
+ * value of link-status is "GOOD". If something fails during or after modeset,
+ * the kernel driver may set this to "BAD" and issue a hotplug uevent. Drivers
+ * should update this value using drm_mode_connector_set_link_status_property().
*
* Connectors also have one standardized atomic property:
*
return -ENOMEM;
dev->mode_config.tile_property = prop;
+ prop = drm_property_create_enum(dev, 0, "link-status",
+ drm_link_status_enum_list,
+ ARRAY_SIZE(drm_link_status_enum_list));
+ if (!prop)
+ return -ENOMEM;
+ dev->mode_config.link_status_property = prop;
+
return 0;
}
}
EXPORT_SYMBOL(drm_mode_connector_update_edid_property);
+/**
+ * drm_mode_connector_set_link_status_property - Set link status property of a connector
+ * @connector: drm connector
+ * @link_status: new value of link status property (0: Good, 1: Bad)
+ *
+ * In usual working scenario, this link status property will always be set to
+ * "GOOD". If something fails during or after a mode set, the kernel driver
+ * may set this link status property to "BAD". The caller then needs to send a
+ * hotplug uevent for userspace to re-check the valid modes through
+ * GET_CONNECTOR_IOCTL and retry modeset.
+ *
+ * Note: Drivers cannot rely on userspace to support this property and
+ * issue a modeset. As such, they may choose to handle issues (like
+ * re-training a link) without userspace's intervention.
+ *
+ * The reason for adding this property is to handle link training failures, but
+ * it is not limited to DP or link training. For example, if we implement
+ * asynchronous setcrtc, this property can be used to report any failures in that.
+ */
+void drm_mode_connector_set_link_status_property(struct drm_connector *connector,
+ uint64_t link_status)
+{
+ struct drm_device *dev = connector->dev;
+
+ drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
+ connector->state->link_status = link_status;
+ drm_modeset_unlock(&dev->mode_config.connection_mutex);
+}
+EXPORT_SYMBOL(drm_mode_connector_set_link_status_property);
+
int drm_mode_connector_set_obj_prop(struct drm_mode_object *obj,
struct drm_property *property,
uint64_t value)
out:
mutex_unlock(&dev->mode_config.mutex);
out_unref:
- drm_connector_unreference(connector);
+ drm_connector_put(connector);
return ret;
}
spin_lock_init(&crtc->commit_lock);
drm_modeset_lock_init(&crtc->mutex);
- ret = drm_mode_object_get(dev, &crtc->base, DRM_MODE_OBJECT_CRTC);
+ ret = drm_mode_object_add(dev, &crtc->base, DRM_MODE_OBJECT_CRTC);
if (ret)
return ret;
drm_for_each_crtc(tmp, crtc->dev) {
if (tmp->primary->fb)
- drm_framebuffer_reference(tmp->primary->fb);
+ drm_framebuffer_get(tmp->primary->fb);
if (tmp->primary->old_fb)
- drm_framebuffer_unreference(tmp->primary->old_fb);
+ drm_framebuffer_put(tmp->primary->old_fb);
tmp->primary->old_fb = NULL;
}
}
fb = crtc->primary->fb;
/* Make refcounting symmetric with the lookup path. */
- drm_framebuffer_reference(fb);
+ drm_framebuffer_get(fb);
} else {
fb = drm_framebuffer_lookup(dev, crtc_req->fb_id);
if (!fb) {
out:
if (fb)
- drm_framebuffer_unreference(fb);
+ drm_framebuffer_put(fb);
if (connector_set) {
for (i = 0; i < crtc_req->count_connectors; i++) {
if (connector_set[i])
- drm_connector_unreference(connector_set[i]);
+ drm_connector_put(connector_set[i]);
}
}
kfree(connector_set);
}
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->encoder == encoder) {
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
return true;
}
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
return false;
}
EXPORT_SYMBOL(drm_helper_encoder_in_use);
if (encoder->crtc != crtc)
continue;
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->encoder != encoder)
continue;
connector->dpms = DRM_MODE_DPMS_OFF;
/* we keep a reference while the encoder is bound */
- drm_connector_unreference(connector);
+ drm_connector_put(connector);
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
}
__drm_helper_disable_unused_functions(dev);
}
count = 0;
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter)
save_connector_encoders[count++] = connector->encoder;
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
save_set.crtc = set->crtc;
save_set.mode = &set->crtc->mode;
for (ro = 0; ro < set->num_connectors; ro++) {
if (set->connectors[ro]->encoder)
continue;
- drm_connector_reference(set->connectors[ro]);
+ drm_connector_get(set->connectors[ro]);
}
/* a) traverse passed in connector list and get encoders for them */
count = 0;
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
const struct drm_connector_helper_funcs *connector_funcs =
connector->helper_private;
connector->encoder = new_encoder;
}
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
if (fail) {
ret = -EINVAL;
}
count = 0;
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
if (!connector->encoder)
continue;
if (new_crtc &&
!drm_encoder_crtc_ok(connector->encoder, new_crtc)) {
ret = -EINVAL;
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
goto fail;
}
if (new_crtc != connector->encoder->crtc) {
connector->base.id, connector->name);
}
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
/* mode_set_base is not a required function */
if (fb_changed && !crtc_funcs->mode_set_base)
}
count = 0;
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter)
connector->encoder = save_connector_encoders[count++];
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
/* after fail drop reference on all unbound connectors in set, let
* bound connectors keep their reference
for (ro = 0; ro < set->num_connectors; ro++) {
if (set->connectors[ro]->encoder)
continue;
- drm_connector_unreference(set->connectors[ro]);
+ drm_connector_put(set->connectors[ro]);
}
/* Try to restore the config */
struct drm_connector_list_iter conn_iter;
struct drm_device *dev = encoder->dev;
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter)
if (connector->encoder == encoder)
if (connector->dpms < dpms)
dpms = connector->dpms;
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
return dpms;
}
struct drm_connector_list_iter conn_iter;
struct drm_device *dev = crtc->dev;
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter)
if (connector->encoder && connector->encoder->crtc == crtc)
if (connector->dpms < dpms)
dpms = connector->dpms;
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
return dpms;
}
void *data, struct drm_file *file_priv);
/* drm_mode_object.c */
-int drm_mode_object_get_reg(struct drm_device *dev,
- struct drm_mode_object *obj,
- uint32_t obj_type,
- bool register_obj,
- void (*obj_free_cb)(struct kref *kref));
+int __drm_mode_object_add(struct drm_device *dev, struct drm_mode_object *obj,
+ uint32_t obj_type, bool register_obj,
+ void (*obj_free_cb)(struct kref *kref));
+int drm_mode_object_add(struct drm_device *dev, struct drm_mode_object *obj,
+ uint32_t obj_type);
void drm_mode_object_register(struct drm_device *dev,
struct drm_mode_object *obj);
-int drm_mode_object_get(struct drm_device *dev,
- struct drm_mode_object *obj, uint32_t obj_type);
struct drm_mode_object *__drm_mode_object_find(struct drm_device *dev,
uint32_t id, uint32_t type);
void drm_mode_object_unregister(struct drm_device *dev,
struct drm_property *property,
uint64_t value);
int drm_connector_create_standard_properties(struct drm_device *dev);
+const char *drm_get_connector_force_name(enum drm_connector_force force);
/* IOCTL */
int drm_mode_connector_property_set_ioctl(struct drm_device *dev,
struct drm_property *property, uint64_t *val);
int drm_mode_atomic_ioctl(struct drm_device *dev,
void *data, struct drm_file *file_priv);
+int drm_atomic_remove_fb(struct drm_framebuffer *fb);
/* drm_plane.c */
*/
int drm_debugfs_cleanup(struct drm_minor *minor)
{
- struct drm_device *dev = minor->dev;
-
if (!minor->debugfs_root)
return 0;
- if (dev->driver->debugfs_cleanup)
- dev->driver->debugfs_cleanup(minor);
-
drm_debugfs_remove_all_files(minor);
debugfs_remove_recursive(minor->debugfs_root);
static int connector_show(struct seq_file *m, void *data)
{
struct drm_connector *connector = m->private;
- const char *status;
-
- switch (connector->force) {
- case DRM_FORCE_ON:
- status = "on\n";
- break;
-
- case DRM_FORCE_ON_DIGITAL:
- status = "digital\n";
- break;
-
- case DRM_FORCE_OFF:
- status = "off\n";
- break;
-
- case DRM_FORCE_UNSPECIFIED:
- status = "unspecified\n";
- break;
-
- default:
- return 0;
- }
- seq_puts(m, status);
+ seq_printf(m, "%s\n", drm_get_connector_force_name(connector->force));
return 0;
}
return "type 2 DVI";
case DRM_DP_DUAL_MODE_TYPE2_HDMI:
return "type 2 HDMI";
+ case DRM_DP_DUAL_MODE_LSPCON:
+ return "lspcon";
default:
WARN_ON(type != DRM_DP_DUAL_MODE_UNKNOWN);
return "unknown";
.unlock_bus = unlock_bus,
};
+static int drm_dp_aux_get_crc(struct drm_dp_aux *aux, u8 *crc)
+{
+ u8 buf, count;
+ int ret;
+
+ ret = drm_dp_dpcd_readb(aux, DP_TEST_SINK, &buf);
+ if (ret < 0)
+ return ret;
+
+ WARN_ON(!(buf & DP_TEST_SINK_START));
+
+ ret = drm_dp_dpcd_readb(aux, DP_TEST_SINK_MISC, &buf);
+ if (ret < 0)
+ return ret;
+
+ count = buf & DP_TEST_COUNT_MASK;
+ if (count == aux->crc_count)
+ return -EAGAIN; /* No CRC yet */
+
+ aux->crc_count = count;
+
+ /*
+ * At DP_TEST_CRC_R_CR, there's 6 bytes containing CRC data, 2 bytes
+ * per component (RGB or CrYCb).
+ */
+ ret = drm_dp_dpcd_read(aux, DP_TEST_CRC_R_CR, crc, 6);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static void drm_dp_aux_crc_work(struct work_struct *work)
+{
+ struct drm_dp_aux *aux = container_of(work, struct drm_dp_aux,
+ crc_work);
+ struct drm_crtc *crtc;
+ u8 crc_bytes[6];
+ uint32_t crcs[3];
+ int ret;
+
+ if (WARN_ON(!aux->crtc))
+ return;
+
+ crtc = aux->crtc;
+ while (crtc->crc.opened) {
+ drm_crtc_wait_one_vblank(crtc);
+ if (!crtc->crc.opened)
+ break;
+
+ ret = drm_dp_aux_get_crc(aux, crc_bytes);
+ if (ret == -EAGAIN) {
+ usleep_range(1000, 2000);
+ ret = drm_dp_aux_get_crc(aux, crc_bytes);
+ }
+
+ if (ret == -EAGAIN) {
+ DRM_DEBUG_KMS("Get CRC failed after retrying: %d\n",
+ ret);
+ continue;
+ } else if (ret) {
+ DRM_DEBUG_KMS("Failed to get a CRC: %d\n", ret);
+ continue;
+ }
+
+ crcs[0] = crc_bytes[0] | crc_bytes[1] << 8;
+ crcs[1] = crc_bytes[2] | crc_bytes[3] << 8;
+ crcs[2] = crc_bytes[4] | crc_bytes[5] << 8;
+ drm_crtc_add_crc_entry(crtc, false, 0, crcs);
+ }
+}
+
/**
* drm_dp_aux_init() - minimally initialise an aux channel
* @aux: DisplayPort AUX channel
void drm_dp_aux_init(struct drm_dp_aux *aux)
{
mutex_init(&aux->hw_mutex);
+ INIT_WORK(&aux->crc_work, drm_dp_aux_crc_work);
aux->ddc.algo = &drm_dp_i2c_algo;
aux->ddc.algo_data = aux;
EXPORT_SYMBOL(drm_dp_psr_setup_time);
#undef PSR_SETUP_TIME
+
+/**
+ * drm_dp_start_crc() - start capture of frame CRCs
+ * @aux: DisplayPort AUX channel
+ * @crtc: CRTC displaying the frames whose CRCs are to be captured
+ *
+ * Returns 0 on success or a negative error code on failure.
+ */
+int drm_dp_start_crc(struct drm_dp_aux *aux, struct drm_crtc *crtc)
+{
+ u8 buf;
+ int ret;
+
+ ret = drm_dp_dpcd_readb(aux, DP_TEST_SINK, &buf);
+ if (ret < 0)
+ return ret;
+
+ ret = drm_dp_dpcd_writeb(aux, DP_TEST_SINK, buf | DP_TEST_SINK_START);
+ if (ret < 0)
+ return ret;
+
+ aux->crc_count = 0;
+ aux->crtc = crtc;
+ schedule_work(&aux->crc_work);
+
+ return 0;
+}
+EXPORT_SYMBOL(drm_dp_start_crc);
+
+/**
+ * drm_dp_stop_crc() - stop capture of frame CRCs
+ * @aux: DisplayPort AUX channel
+ *
+ * Returns 0 on success or a negative error code on failure.
+ */
+int drm_dp_stop_crc(struct drm_dp_aux *aux)
+{
+ u8 buf;
+ int ret;
+
+ ret = drm_dp_dpcd_readb(aux, DP_TEST_SINK, &buf);
+ if (ret < 0)
+ return ret;
+
+ ret = drm_dp_dpcd_writeb(aux, DP_TEST_SINK, buf & ~DP_TEST_SINK_START);
+ if (ret < 0)
+ return ret;
+
+ flush_work(&aux->crc_work);
+ aux->crtc = NULL;
+
+ return 0;
+}
+EXPORT_SYMBOL(drm_dp_stop_crc);
#include <drm/drm_edid.h>
#include <drm/drm_encoder.h>
#include <drm/drm_displayid.h>
+#include <drm/drm_scdc_helper.h>
#include "drm_crtc_internal.h"
csum = drm_edid_block_checksum(raw_edid);
if (csum) {
- if (print_bad_edid) {
- DRM_ERROR("EDID checksum is invalid, remainder is %d\n", csum);
- }
-
if (edid_corrupt)
*edid_corrupt = true;
/* allow CEA to slide through, switches mangle this */
- if (raw_edid[0] != 0x02)
+ if (raw_edid[0] == CEA_EXT) {
+ DRM_DEBUG("EDID checksum is invalid, remainder is %d\n", csum);
+ DRM_DEBUG("Assuming a KVM switch modified the CEA block but left the original checksum\n");
+ } else {
+ if (print_bad_edid)
+ DRM_NOTE("EDID checksum is invalid, remainder is %d\n", csum);
+
goto bad;
+ }
}
/* per-block-type checks */
switch (raw_edid[0]) {
case 0: /* base */
if (edid->version != 1) {
- DRM_ERROR("EDID has major version %d, instead of 1\n", edid->version);
+ DRM_NOTE("EDID has major version %d, instead of 1\n", edid->version);
goto bad;
}
bad:
if (print_bad_edid) {
if (drm_edid_is_zero(raw_edid, EDID_LENGTH)) {
- printk(KERN_ERR "EDID block is all zeroes\n");
+ pr_notice("EDID block is all zeroes\n");
} else {
- printk(KERN_ERR "Raw EDID:\n");
- print_hex_dump(KERN_ERR, " \t", DUMP_PREFIX_NONE, 16, 1,
- raw_edid, EDID_LENGTH, false);
+ pr_notice("Raw EDID:\n");
+ print_hex_dump(KERN_NOTICE,
+ " \t", DUMP_PREFIX_NONE, 16, 1,
+ raw_edid, EDID_LENGTH, false);
}
}
return false;
{
struct edid *edid;
- if (!drm_probe_ddc(adapter))
+ if (connector->force == DRM_FORCE_OFF)
+ return NULL;
+
+ if (connector->force == DRM_FORCE_UNSPECIFIED && !drm_probe_ddc(adapter))
return NULL;
edid = drm_do_get_edid(connector, drm_do_probe_ddc_edid, adapter);
return hdmi_id == HDMI_IEEE_OUI;
}
+static bool cea_db_is_hdmi_forum_vsdb(const u8 *db)
+{
+ unsigned int oui;
+
+ if (cea_db_tag(db) != VENDOR_BLOCK)
+ return false;
+
+ if (cea_db_payload_len(db) < 7)
+ return false;
+
+ oui = db[3] << 16 | db[2] << 8 | db[1];
+
+ return oui == HDMI_FORUM_IEEE_OUI;
+}
+
#define for_each_cea_db(cea, i, start, end) \
for ((i) = (start); (i) < (end) && (i) + cea_db_payload_len(&(cea)[(i)]) < (end); (i) += cea_db_payload_len(&(cea)[(i)]) + 1)
connector->video_latency[1] = 0;
connector->audio_latency[1] = 0;
+ if (!edid)
+ return;
+
cea = drm_find_cea_extension(edid);
if (!cea) {
DRM_DEBUG_KMS("ELD: no CEA Extension found\n");
}
EXPORT_SYMBOL(drm_default_rgb_quant_range);
+static void drm_parse_hdmi_forum_vsdb(struct drm_connector *connector,
+ const u8 *hf_vsdb)
+{
+ struct drm_display_info *display = &connector->display_info;
+ struct drm_hdmi_info *hdmi = &display->hdmi;
+
+ if (hf_vsdb[6] & 0x80) {
+ hdmi->scdc.supported = true;
+ if (hf_vsdb[6] & 0x40)
+ hdmi->scdc.read_request = true;
+ }
+
+ /*
+ * All HDMI 2.0 monitors must support scrambling at rates > 340 MHz.
+ * And as per the spec, three factors confirm this:
+ * * Availability of a HF-VSDB block in EDID (check)
+ * * Non zero Max_TMDS_Char_Rate filed in HF-VSDB (let's check)
+ * * SCDC support available (let's check)
+ * Lets check it out.
+ */
+
+ if (hf_vsdb[5]) {
+ /* max clock is 5000 KHz times block value */
+ u32 max_tmds_clock = hf_vsdb[5] * 5000;
+ struct drm_scdc *scdc = &hdmi->scdc;
+
+ if (max_tmds_clock > 340000) {
+ display->max_tmds_clock = max_tmds_clock;
+ DRM_DEBUG_KMS("HF-VSDB: max TMDS clock %d kHz\n",
+ display->max_tmds_clock);
+ }
+
+ if (scdc->supported) {
+ scdc->scrambling.supported = true;
+
+ /* Few sinks support scrambling for cloks < 340M */
+ if ((hf_vsdb[6] & 0x8))
+ scdc->scrambling.low_rates = true;
+ }
+ }
+}
+
static void drm_parse_hdmi_deep_color_info(struct drm_connector *connector,
const u8 *hdmi)
{
if (cea_db_is_hdmi_vsdb(db))
drm_parse_hdmi_vsdb_video(connector, db);
+ if (cea_db_is_hdmi_forum_vsdb(db))
+ drm_parse_hdmi_forum_vsdb(connector, db);
}
}
csum += displayid[i];
}
if (csum) {
- DRM_ERROR("DisplayID checksum invalid, remainder is %d\n", csum);
+ DRM_NOTE("DisplayID checksum invalid, remainder is %d\n", csum);
return -EINVAL;
}
return 0;
return edid;
}
-int drm_load_edid_firmware(struct drm_connector *connector)
+struct edid *drm_load_edid_firmware(struct drm_connector *connector)
{
const char *connector_name = connector->name;
char *edidname, *last, *colon, *fwstr, *edidstr, *fallback = NULL;
- int ret;
struct edid *edid;
if (edid_firmware[0] == '\0')
- return 0;
+ return ERR_PTR(-ENOENT);
/*
* If there are multiple edid files specified and separated
if (!edidname) {
if (!fallback) {
kfree(fwstr);
- return 0;
+ return ERR_PTR(-ENOENT);
}
edidname = fallback;
}
edid = edid_load(connector, edidname, connector_name);
kfree(fwstr);
- if (IS_ERR_OR_NULL(edid))
- return 0;
-
- drm_mode_connector_update_edid_property(connector, edid);
- ret = drm_add_edid_modes(connector, edid);
- drm_edid_to_eld(connector, edid);
- kfree(edid);
-
- return ret;
+ return edid;
}
{
int ret;
- ret = drm_mode_object_get(dev, &encoder->base, DRM_MODE_OBJECT_ENCODER);
+ ret = drm_mode_object_add(dev, &encoder->base, DRM_MODE_OBJECT_ENCODER);
if (ret)
return ret;
/* For atomic drivers only state objects are synchronously updated and
* protected by modeset locks, so check those first. */
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
if (!connector->state)
continue;
if (connector->state->best_encoder != encoder)
continue;
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
return connector->state->crtc;
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
/* Don't return stale data (e.g. pending async disable). */
if (uses_atomic)
for (i = 0; i < 4; i++) {
if (fb_cma->obj[i])
- drm_gem_object_unreference_unlocked(&fb_cma->obj[i]->base);
+ drm_gem_object_put_unlocked(&fb_cma->obj[i]->base);
}
drm_framebuffer_cleanup(fb);
if (!obj) {
dev_err(dev->dev, "Failed to lookup GEM object\n");
ret = -ENXIO;
- goto err_gem_object_unreference;
+ goto err_gem_object_put;
}
min_size = (height - 1) * mode_cmd->pitches[i]
+ mode_cmd->offsets[i];
if (obj->size < min_size) {
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
ret = -EINVAL;
- goto err_gem_object_unreference;
+ goto err_gem_object_put;
}
objs[i] = to_drm_gem_cma_obj(obj);
}
fb_cma = drm_fb_cma_alloc(dev, mode_cmd, objs, i, funcs);
if (IS_ERR(fb_cma)) {
ret = PTR_ERR(fb_cma);
- goto err_gem_object_unreference;
+ goto err_gem_object_put;
}
return &fb_cma->fb;
-err_gem_object_unreference:
+err_gem_object_put:
for (i--; i >= 0; i--)
- drm_gem_object_unreference_unlocked(&objs[i]->base);
+ drm_gem_object_put_unlocked(&objs[i]->base);
return ERR_PTR(ret);
}
EXPORT_SYMBOL_GPL(drm_fb_cma_create_with_funcs);
err_cma_destroy:
drm_framebuffer_remove(&fbdev_cma->fb->fb);
err_fb_info_destroy:
- drm_fb_helper_release_fbi(helper);
+ drm_fb_helper_fini(helper);
err_gem_free_object:
- drm_gem_object_unreference_unlocked(&obj->base);
+ drm_gem_object_put_unlocked(&obj->base);
return ret;
}
* drm_fbdev_cma_init() - Allocate and initializes a drm_fbdev_cma struct
* @dev: DRM device
* @preferred_bpp: Preferred bits per pixel for the device
- * @num_crtc: Number of CRTCs
* @max_conn_count: Maximum number of connectors
*
* Returns a newly allocated drm_fbdev_cma struct or a ERR_PTR.
drm_fb_helper_unregister_fbi(&fbdev_cma->fb_helper);
if (fbdev_cma->fb_helper.fbdev)
drm_fbdev_cma_defio_fini(fbdev_cma->fb_helper.fbdev);
- drm_fb_helper_release_fbi(&fbdev_cma->fb_helper);
if (fbdev_cma->fb)
drm_framebuffer_remove(&fbdev_cma->fb->fb);
MODULE_PARM_DESC(fbdev_emulation,
"Enable legacy fbdev emulation [default=true]");
+static int drm_fbdev_overalloc = CONFIG_DRM_FBDEV_OVERALLOC;
+module_param(drm_fbdev_overalloc, int, 0444);
+MODULE_PARM_DESC(drm_fbdev_overalloc,
+ "Overallocation of the fbdev buffer (%) [default="
+ __MODULE_STRING(CONFIG_DRM_FBDEV_OVERALLOC) "]");
+
static LIST_HEAD(kernel_fb_helper_list);
static DEFINE_MUTEX(kernel_fb_helper_lock);
* drm_fb_helper_init(), drm_fb_helper_single_add_all_connectors() and
* drm_fb_helper_initial_config(). Drivers with fancier requirements than the
* default behaviour can override the third step with their own code.
- * Teardown is done with drm_fb_helper_fini().
+ * Teardown is done with drm_fb_helper_fini() after the fbdev device is
+ * unregisters using drm_fb_helper_unregister_fbi().
*
* At runtime drivers should restore the fbdev console by calling
* drm_fb_helper_restore_fbdev_mode_unlocked() from their &drm_driver.lastclose
return 0;
mutex_lock(&dev->mode_config.mutex);
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
ret = drm_fb_helper_add_one_connector(fb_helper, connector);
struct drm_fb_helper_connector *fb_helper_connector =
fb_helper->connector_info[i];
- drm_connector_unreference(fb_helper_connector->connector);
+ drm_connector_put(fb_helper_connector->connector);
kfree(fb_helper_connector);
fb_helper->connector_info[i] = NULL;
}
fb_helper->connector_count = 0;
out:
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
mutex_unlock(&dev->mode_config.mutex);
return ret;
if (!fb_helper_connector)
return -ENOMEM;
- drm_connector_reference(connector);
+ drm_connector_get(connector);
fb_helper_connector->connector = connector;
fb_helper->connector_info[fb_helper->connector_count++] = fb_helper_connector;
return 0;
if (i == fb_helper->connector_count)
return -EINVAL;
fb_helper_connector = fb_helper->connector_info[i];
- drm_connector_unreference(fb_helper_connector->connector);
+ drm_connector_put(fb_helper_connector->connector);
for (j = i + 1; j < fb_helper->connector_count; j++) {
fb_helper->connector_info[j - 1] = fb_helper->connector_info[j];
int i;
for (i = 0; i < modeset->num_connectors; i++) {
- drm_connector_unreference(modeset->connectors[i]);
+ drm_connector_put(modeset->connectors[i]);
modeset->connectors[i] = NULL;
}
modeset->num_connectors = 0;
int i;
for (i = 0; i < helper->connector_count; i++) {
- drm_connector_unreference(helper->connector_info[i]->connector);
+ drm_connector_put(helper->connector_info[i]->connector);
kfree(helper->connector_info[i]);
}
kfree(helper->connector_info);
EXPORT_SYMBOL(drm_fb_helper_prepare);
/**
- * drm_fb_helper_init - initialize a drm_fb_helper structure
+ * drm_fb_helper_init - initialize a &struct drm_fb_helper
* @dev: drm device
* @fb_helper: driver-allocated fbdev helper structure to initialize
* @max_conn_count: max connector count
* @fb_helper: driver-allocated fbdev helper
*
* A helper to alloc fb_info and the members cmap and apertures. Called
- * by the driver within the fb_probe fb_helper callback function.
+ * by the driver within the fb_probe fb_helper callback function. Drivers do not
+ * need to release the allocated fb_info structure themselves, this is
+ * automatically done when calling drm_fb_helper_fini().
*
* RETURNS:
* fb_info pointer if things went okay, pointer containing error code
* @fb_helper: driver-allocated fbdev helper
*
* A wrapper around unregister_framebuffer, to release the fb_info
- * framebuffer device
+ * framebuffer device. This must be called before releasing all resources for
+ * @fb_helper by calling drm_fb_helper_fini().
*/
void drm_fb_helper_unregister_fbi(struct drm_fb_helper *fb_helper)
{
EXPORT_SYMBOL(drm_fb_helper_unregister_fbi);
/**
- * drm_fb_helper_release_fbi - dealloc fb_info and its members
+ * drm_fb_helper_fini - finialize a &struct drm_fb_helper
* @fb_helper: driver-allocated fbdev helper
*
- * A helper to free memory taken by fb_info and the members cmap and
- * apertures
+ * This cleans up all remaining resources associated with @fb_helper. Must be
+ * called after drm_fb_helper_unlink_fbi() was called.
*/
-void drm_fb_helper_release_fbi(struct drm_fb_helper *fb_helper)
+void drm_fb_helper_fini(struct drm_fb_helper *fb_helper)
{
- if (fb_helper) {
- struct fb_info *info = fb_helper->fbdev;
+ struct fb_info *info;
- if (info) {
- if (info->cmap.len)
- fb_dealloc_cmap(&info->cmap);
- framebuffer_release(info);
- }
+ if (!drm_fbdev_emulation || !fb_helper)
+ return;
- fb_helper->fbdev = NULL;
+ info = fb_helper->fbdev;
+ if (info) {
+ if (info->cmap.len)
+ fb_dealloc_cmap(&info->cmap);
+ framebuffer_release(info);
}
-}
-EXPORT_SYMBOL(drm_fb_helper_release_fbi);
-
-void drm_fb_helper_fini(struct drm_fb_helper *fb_helper)
-{
- if (!drm_fbdev_emulation)
- return;
+ fb_helper->fbdev = NULL;
cancel_work_sync(&fb_helper->resume_work);
cancel_work_sync(&fb_helper->dirty_work);
}
EXPORT_SYMBOL(drm_fb_helper_setcmap);
+/**
+ * drm_fb_helper_ioctl - legacy ioctl implementation
+ * @info: fbdev registered by the helper
+ * @cmd: ioctl command
+ * @arg: ioctl argument
+ *
+ * A helper to implement the standard fbdev ioctl. Only
+ * FBIO_WAITFORVSYNC is implemented for now.
+ */
+int drm_fb_helper_ioctl(struct fb_info *info, unsigned int cmd,
+ unsigned long arg)
+{
+ struct drm_fb_helper *fb_helper = info->par;
+ struct drm_device *dev = fb_helper->dev;
+ struct drm_mode_set *mode_set;
+ struct drm_crtc *crtc;
+ int ret = 0;
+
+ mutex_lock(&dev->mode_config.mutex);
+ if (!drm_fb_helper_is_bound(fb_helper)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ switch (cmd) {
+ case FBIO_WAITFORVSYNC:
+ /*
+ * Only consider the first CRTC.
+ *
+ * This ioctl is supposed to take the CRTC number as
+ * an argument, but in fbdev times, what that number
+ * was supposed to be was quite unclear, different
+ * drivers were passing that argument differently
+ * (some by reference, some by value), and most of the
+ * userspace applications were just hardcoding 0 as an
+ * argument.
+ *
+ * The first CRTC should be the integrated panel on
+ * most drivers, so this is the best choice we can
+ * make. If we're not smart enough here, one should
+ * just consider switch the userspace to KMS.
+ */
+ mode_set = &fb_helper->crtc_info[0].mode_set;
+ crtc = mode_set->crtc;
+
+ /*
+ * Only wait for a vblank event if the CRTC is
+ * enabled, otherwise just don't do anythintg,
+ * not even report an error.
+ */
+ ret = drm_crtc_vblank_get(crtc);
+ if (!ret) {
+ drm_crtc_wait_one_vblank(crtc);
+ drm_crtc_vblank_put(crtc);
+ }
+
+ ret = 0;
+ goto unlock;
+ default:
+ ret = -ENOTTY;
+ }
+
+unlock:
+ mutex_unlock(&dev->mode_config.mutex);
+ return ret;
+}
+EXPORT_SYMBOL(drm_fb_helper_ioctl);
+
/**
* drm_fb_helper_check_var - implementation for &fb_ops.fb_check_var
* @var: screeninfo to check
sizes.fb_height = sizes.surface_height = 768;
}
+ /* Handle our overallocation */
+ sizes.surface_height *= drm_fbdev_overalloc;
+ sizes.surface_height /= 100;
+
/* push down into drivers */
ret = (*fb_helper->funcs->fb_probe)(fb_helper, &sizes);
if (ret < 0)
fb_crtc->y = offset->y;
modeset->mode = drm_mode_duplicate(dev,
fb_crtc->desired_mode);
- drm_connector_reference(connector);
+ drm_connector_get(connector);
modeset->connectors[modeset->num_connectors++] = connector;
modeset->fb = fb_helper->fb;
modeset->x = offset->x;
/*
- * \file drm_fops.c
- * File operations for DRM
- *
* \author Rickard E. (Rik) Faith <faith@valinux.com>
* \author Daryll Strauss <daryll@valinux.com>
* \author Gareth Hughes <gareth@valinux.com>
* OTHER DEALINGS IN THE SOFTWARE.
*/
-#include <drm/drmP.h>
#include <linux/poll.h>
#include <linux/slab.h>
#include <linux/module.h>
+
+#include <drm/drm_file.h>
+#include <drm/drmP.h>
+
#include "drm_legacy.h"
#include "drm_internal.h"
#include "drm_crtc_internal.h"
*
* Drivers must define the file operations structure that forms the DRM
* userspace API entry point, even though most of those operations are
- * implemented in the DRM core. The mandatory functions are drm_open(),
+ * implemented in the DRM core. The resulting &struct file_operations must be
+ * stored in the &drm_driver.fops field. The mandatory functions are drm_open(),
* drm_read(), drm_ioctl() and drm_compat_ioctl() if CONFIG_COMPAT is enabled
- * (note that drm_compat_ioctl will be NULL if CONFIG_COMPAT=n). Drivers which
- * implement private ioctls that require 32/64 bit compatibility support must
- * provide their own .compat_ioctl() handler that processes private ioctls and
- * calls drm_compat_ioctl() for core ioctls.
+ * Note that drm_compat_ioctl will be NULL if CONFIG_COMPAT=n, so there's no
+ * need to sprinkle #ifdef into the code. Drivers which implement private ioctls
+ * that require 32/64 bit compatibility support must provide their own
+ * &file_operations.compat_ioctl handler that processes private ioctls and calls
+ * drm_compat_ioctl() for core ioctls.
*
* In addition drm_read() and drm_poll() provide support for DRM events. DRM
* events are a generic and extensible means to send asynchronous events to
* page flip completions by the KMS API. But drivers can also use it for their
* own needs, e.g. to signal completion of rendering.
*
+ * For the driver-side event interface see drm_event_reserve_init() and
+ * drm_send_event() as the main starting points.
+ *
* The memory mapping implementation will vary depending on how the driver
* manages memory. Legacy drivers will use the deprecated drm_legacy_mmap()
* function, modern drivers should use one of the provided memory-manager
- * specific implementations. For GEM-based drivers this is drm_gem_mmap().
+ * specific implementations. For GEM-based drivers this is drm_gem_mmap(), and
+ * for drivers which use the CMA GEM helpers it's drm_gem_cma_mmap().
*
* No other file operations are supported by the DRM userspace API. Overall the
* following is an example #file_operations structure::
* .llseek = no_llseek,
* .mmap = drm_gem_mmap,
* };
+ *
+ * For plain GEM based drivers there is the DEFINE_DRM_GEM_FOPS() macro, and for
+ * CMA based drivers there is the DEFINE_DRM_GEM_CMA_FOPS() macro to make this
+ * simpler.
*/
static int drm_open_helper(struct file *filp, struct drm_minor *minor);
* @inode: device inode
* @filp: file pointer.
*
- * This function must be used by drivers as their .open() #file_operations
- * method. It looks up the correct DRM device and instantiates all the per-file
- * resources for it.
+ * This function must be used by drivers as their &file_operations.open method.
+ * It looks up the correct DRM device and instantiates all the per-file
+ * resources for it. It also calls the &drm_driver.open driver callback.
*
* RETURNS:
*
spin_unlock_irqrestore(&dev->event_lock, flags);
}
-/*
- * drm_legacy_dev_reinit
- *
- * Reinitializes a legacy/ums drm device in it's lastclose function.
- */
static void drm_legacy_dev_reinit(struct drm_device *dev)
{
if (dev->irq_enabled)
DRM_DEBUG("lastclose completed\n");
}
-/*
- * Take down the DRM device.
- *
- * \param dev DRM device structure.
- *
- * Frees every resource in \p dev.
- *
- * \sa drm_device
- */
void drm_lastclose(struct drm_device * dev)
{
DRM_DEBUG("\n");
* @inode: device inode
* @filp: file pointer.
*
- * This function must be used by drivers as their .release() #file_operations
- * method. It frees any resources associated with the open file, and if this is
- * the last open file for the DRM device also proceeds to call drm_lastclose().
+ * This function must be used by drivers as their &file_operations.release
+ * method. It frees any resources associated with the open file, and calls the
+ * &drm_driver.preclose and &drm_driver.lastclose driver callbacks. If this is
+ * the last open file for the DRM device also proceeds to call the
+ * &drm_driver.lastclose driver callback.
*
* RETURNS:
*
* @count: count in bytes to read
* @offset: offset to read
*
- * This function must be used by drivers as their .read() #file_operations
+ * This function must be used by drivers as their &file_operations.read
* method iff they use DRM events for asynchronous signalling to userspace.
* Since events are used by the KMS API for vblank and page flip completion this
* means all modern display drivers must use it.
*
- * @offset is ignore, DRM events are read like a pipe. Therefore drivers also
- * must set the .llseek() #file_operation to no_llseek(). Polling support is
+ * @offset is ignored, DRM events are read like a pipe. Therefore drivers also
+ * must set the &file_operation.llseek to no_llseek(). Polling support is
* provided by drm_poll().
*
* This function will only ever read a full event. Therefore userspace must
* @filp: file pointer
* @wait: poll waiter table
*
- * This function must be used by drivers as their .read() #file_operations
- * method iff they use DRM events for asynchronous signalling to userspace.
- * Since events are used by the KMS API for vblank and page flip completion this
- * means all modern display drivers must use it.
+ * This function must be used by drivers as their &file_operations.read method
+ * iff they use DRM events for asynchronous signalling to userspace. Since
+ * events are used by the KMS API for vblank and page flip completion this means
+ * all modern display drivers must use it.
*
* See also drm_read().
*
* @p: tracking structure for the pending event
*
* This function frees the event @p initialized with drm_event_reserve_init()
- * and releases any allocated space.
+ * and releases any allocated space. It is used to cancel an event when the
+ * nonblocking operation could not be submitted and needed to be aborted.
*/
void drm_event_cancel_free(struct drm_device *dev,
struct drm_pending_event *p)
{ .format = DRM_FORMAT_XBGR8888, .depth = 24, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
{ .format = DRM_FORMAT_RGBX8888, .depth = 24, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
{ .format = DRM_FORMAT_BGRX8888, .depth = 24, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+ { .format = DRM_FORMAT_RGB565_A8, .depth = 24, .num_planes = 2, .cpp = { 2, 1, 0 }, .hsub = 1, .vsub = 1 },
+ { .format = DRM_FORMAT_BGR565_A8, .depth = 24, .num_planes = 2, .cpp = { 2, 1, 0 }, .hsub = 1, .vsub = 1 },
{ .format = DRM_FORMAT_XRGB2101010, .depth = 30, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
{ .format = DRM_FORMAT_XBGR2101010, .depth = 30, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
{ .format = DRM_FORMAT_RGBX1010102, .depth = 30, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
{ .format = DRM_FORMAT_ABGR8888, .depth = 32, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
{ .format = DRM_FORMAT_RGBA8888, .depth = 32, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
{ .format = DRM_FORMAT_BGRA8888, .depth = 32, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+ { .format = DRM_FORMAT_RGB888_A8, .depth = 32, .num_planes = 2, .cpp = { 3, 1, 0 }, .hsub = 1, .vsub = 1 },
+ { .format = DRM_FORMAT_BGR888_A8, .depth = 32, .num_planes = 2, .cpp = { 3, 1, 0 }, .hsub = 1, .vsub = 1 },
+ { .format = DRM_FORMAT_XRGB8888_A8, .depth = 32, .num_planes = 2, .cpp = { 4, 1, 0 }, .hsub = 1, .vsub = 1 },
+ { .format = DRM_FORMAT_XBGR8888_A8, .depth = 32, .num_planes = 2, .cpp = { 4, 1, 0 }, .hsub = 1, .vsub = 1 },
+ { .format = DRM_FORMAT_RGBX8888_A8, .depth = 32, .num_planes = 2, .cpp = { 4, 1, 0 }, .hsub = 1, .vsub = 1 },
+ { .format = DRM_FORMAT_BGRX8888_A8, .depth = 32, .num_planes = 2, .cpp = { 4, 1, 0 }, .hsub = 1, .vsub = 1 },
{ .format = DRM_FORMAT_YUV410, .depth = 0, .num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 4 },
{ .format = DRM_FORMAT_YVU410, .depth = 0, .num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 4 },
{ .format = DRM_FORMAT_YUV411, .depth = 0, .num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 1 },
* metadata fields.
*
* The lifetime of a drm framebuffer is controlled with a reference count,
- * drivers can grab additional references with drm_framebuffer_reference() and
- * drop them again with drm_framebuffer_unreference(). For driver-private
- * framebuffers for which the last reference is never dropped (e.g. for the
- * fbdev framebuffer when the struct &struct drm_framebuffer is embedded into
- * the fbdev helper struct) drivers can manually clean up a framebuffer at
- * module unload time with drm_framebuffer_unregister_private(). But doing this
- * is not recommended, and it's better to have a normal free-standing &struct
+ * drivers can grab additional references with drm_framebuffer_get() and drop
+ * them again with drm_framebuffer_put(). For driver-private framebuffers for
+ * which the last reference is never dropped (e.g. for the fbdev framebuffer
+ * when the struct &struct drm_framebuffer is embedded into the fbdev helper
+ * struct) drivers can manually clean up a framebuffer at module unload time
+ * with drm_framebuffer_unregister_private(). But doing this is not
+ * recommended, and it's better to have a normal free-standing &struct
* drm_framebuffer.
*/
mutex_unlock(&file_priv->fbs_lock);
/* drop the reference we picked up in framebuffer lookup */
- drm_framebuffer_unreference(fb);
+ drm_framebuffer_put(fb);
/*
* we now own the reference that was stored in the fbs list
flush_work(&arg.work);
destroy_work_on_stack(&arg.work);
} else
- drm_framebuffer_unreference(fb);
+ drm_framebuffer_put(fb);
return 0;
fail_unref:
- drm_framebuffer_unreference(fb);
+ drm_framebuffer_put(fb);
return -ENOENT;
}
ret = -ENODEV;
}
- drm_framebuffer_unreference(fb);
+ drm_framebuffer_put(fb);
return ret;
}
out_err2:
kfree(clips);
out_err1:
- drm_framebuffer_unreference(fb);
+ drm_framebuffer_put(fb);
return ret;
}
list_del_init(&fb->filp_head);
/* This drops the fpriv->fbs reference. */
- drm_framebuffer_unreference(fb);
+ drm_framebuffer_put(fb);
}
}
fb->funcs = funcs;
- ret = drm_mode_object_get_reg(dev, &fb->base, DRM_MODE_OBJECT_FB,
- false, drm_framebuffer_free);
+ ret = __drm_mode_object_add(dev, &fb->base, DRM_MODE_OBJECT_FB,
+ false, drm_framebuffer_free);
if (ret)
goto out;
*
* If successful, this grabs an additional reference to the framebuffer -
* callers need to make sure to eventually unreference the returned framebuffer
- * again, using @drm_framebuffer_unreference.
+ * again, using drm_framebuffer_put().
*/
struct drm_framebuffer *drm_framebuffer_lookup(struct drm_device *dev,
uint32_t id)
*
* NOTE: This function is deprecated. For driver-private framebuffers it is not
* recommended to embed a framebuffer struct info fbdev struct, instead, a
- * framebuffer pointer is preferred and drm_framebuffer_unreference() should be
- * called when the framebuffer is to be cleaned up.
+ * framebuffer pointer is preferred and drm_framebuffer_put() should be called
+ * when the framebuffer is to be cleaned up.
*/
void drm_framebuffer_unregister_private(struct drm_framebuffer *fb)
{
* in this manner.
*/
if (drm_framebuffer_read_refcount(fb) > 1) {
+ if (drm_drv_uses_atomic_modeset(dev)) {
+ int ret = drm_atomic_remove_fb(fb);
+ WARN(ret, "atomic remove_fb failed with %i\n", ret);
+ goto out;
+ }
+
drm_modeset_lock_all(dev);
/* remove from any CRTC */
drm_for_each_crtc(crtc, dev) {
drm_modeset_unlock_all(dev);
}
- drm_framebuffer_unreference(fb);
+out:
+ drm_framebuffer_put(fb);
}
EXPORT_SYMBOL(drm_framebuffer_remove);
}
static void
-drm_gem_object_handle_unreference_unlocked(struct drm_gem_object *obj)
+drm_gem_object_handle_put_unlocked(struct drm_gem_object *obj)
{
struct drm_device *dev = obj->dev;
bool final = false;
mutex_unlock(&dev->object_name_lock);
if (final)
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
}
/*
if (dev->driver->gem_close_object)
dev->driver->gem_close_object(obj, file_priv);
- drm_gem_object_handle_unreference_unlocked(obj);
+ drm_gem_object_handle_put_unlocked(obj);
return 0;
}
WARN_ON(!mutex_is_locked(&dev->object_name_lock));
if (obj->handle_count++ == 0)
- drm_gem_object_reference(obj);
+ drm_gem_object_get(obj);
/*
* Get the user-visible handle using idr. Preload and perform
idr_remove(&file_priv->object_idr, handle);
spin_unlock(&file_priv->table_lock);
err_unref:
- drm_gem_object_handle_unreference_unlocked(obj);
+ drm_gem_object_handle_put_unlocked(obj);
return ret;
}
/* Check if we currently have a reference on the object */
obj = idr_find(&filp->object_idr, handle);
if (obj)
- drm_gem_object_reference(obj);
+ drm_gem_object_get(obj);
spin_unlock(&filp->table_lock);
err:
mutex_unlock(&dev->object_name_lock);
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
return ret;
}
mutex_lock(&dev->object_name_lock);
obj = idr_find(&dev->object_name_idr, (int) args->name);
if (obj) {
- drm_gem_object_reference(obj);
+ drm_gem_object_get(obj);
} else {
mutex_unlock(&dev->object_name_lock);
return -ENOENT;
/* drm_gem_handle_create_tail unlocks dev->object_name_lock. */
ret = drm_gem_handle_create_tail(file_priv, obj, &handle);
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
if (ret)
return ret;
EXPORT_SYMBOL(drm_gem_object_free);
/**
- * drm_gem_object_unreference_unlocked - release a GEM BO reference
+ * drm_gem_object_put_unlocked - drop a GEM buffer object reference
* @obj: GEM buffer object
*
* This releases a reference to @obj. Callers must not hold the
* &drm_device.struct_mutex lock when calling this function.
*
- * See also __drm_gem_object_unreference().
+ * See also __drm_gem_object_put().
*/
void
-drm_gem_object_unreference_unlocked(struct drm_gem_object *obj)
+drm_gem_object_put_unlocked(struct drm_gem_object *obj)
{
struct drm_device *dev;
&dev->struct_mutex))
mutex_unlock(&dev->struct_mutex);
}
-EXPORT_SYMBOL(drm_gem_object_unreference_unlocked);
+EXPORT_SYMBOL(drm_gem_object_put_unlocked);
/**
- * drm_gem_object_unreference - release a GEM BO reference
+ * drm_gem_object_put - release a GEM buffer object reference
* @obj: GEM buffer object
*
* This releases a reference to @obj. Callers must hold the
* driver doesn't use &drm_device.struct_mutex for anything.
*
* For drivers not encumbered with legacy locking use
- * drm_gem_object_unreference_unlocked() instead.
+ * drm_gem_object_put_unlocked() instead.
*/
void
-drm_gem_object_unreference(struct drm_gem_object *obj)
+drm_gem_object_put(struct drm_gem_object *obj)
{
if (obj) {
WARN_ON(!mutex_is_locked(&obj->dev->struct_mutex));
kref_put(&obj->refcount, drm_gem_object_free);
}
}
-EXPORT_SYMBOL(drm_gem_object_unreference);
+EXPORT_SYMBOL(drm_gem_object_put);
/**
* drm_gem_vm_open - vma->ops->open implementation for GEM
{
struct drm_gem_object *obj = vma->vm_private_data;
- drm_gem_object_reference(obj);
+ drm_gem_object_get(obj);
}
EXPORT_SYMBOL(drm_gem_vm_open);
{
struct drm_gem_object *obj = vma->vm_private_data;
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
}
EXPORT_SYMBOL(drm_gem_vm_close);
* (which should happen whether the vma was created by this call, or
* by a vm_open due to mremap or partial unmap or whatever).
*/
- drm_gem_object_reference(obj);
+ drm_gem_object_get(obj);
return 0;
}
return -EINVAL;
if (!drm_vma_node_is_allowed(node, priv)) {
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
return -EACCES;
}
ret = drm_gem_mmap_obj(obj, drm_vma_node_size(node) << PAGE_SHIFT,
vma);
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
return ret;
}
return cma_obj;
error:
- drm_gem_object_unreference_unlocked(&cma_obj->base);
+ drm_gem_object_put_unlocked(&cma_obj->base);
return ERR_PTR(ret);
}
EXPORT_SYMBOL_GPL(drm_gem_cma_create);
*/
ret = drm_gem_handle_create(file_priv, gem_obj, handle);
/* drop reference from allocate - handle holds it now. */
- drm_gem_object_unreference_unlocked(gem_obj);
+ drm_gem_object_put_unlocked(gem_obj);
if (ret)
return ERR_PTR(ret);
*offset = drm_vma_node_offset_addr(&gem_obj->vma_node);
- drm_gem_object_unreference_unlocked(gem_obj);
+ drm_gem_object_put_unlocked(gem_obj);
return 0;
}
* as their ->mmap() handler in the DRM device file's file_operations
* structure.
*
+ * Instead of directly referencing this function, drivers should use the
+ * DEFINE_DRM_GEM_CMA_FOPS().macro.
+ *
* Returns:
* 0 on success or a negative error code on failure.
*/
return -EINVAL;
if (!drm_vma_node_is_allowed(node, priv)) {
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
return -EACCES;
}
cma_obj = to_drm_gem_cma_obj(obj);
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
return cma_obj->vaddr ? (unsigned long)cma_obj->vaddr : -EINVAL;
}
#define DRM_IF_MAJOR 1
#define DRM_IF_MINOR 4
-/* drm_fops.c */
+/* drm_file.c */
extern struct mutex drm_global_mutex;
void drm_lastclose(struct drm_device *dev);
m32.handle = (unsigned long)handle;
if (m32.handle != (unsigned long)handle)
- printk_ratelimited(KERN_ERR "compat_drm_addmap truncated handle"
- " %p for type %d offset %x\n",
+ pr_err_ratelimited("compat_drm_addmap truncated handle %p for type %d offset %x\n",
handle, m32.type, m32.offset);
if (copy_to_user(argp, &m32, sizeof(m32)))
write_sequnlock(&vblank->seqlock);
}
+/*
+ * "No hw counter" fallback implementation of .get_vblank_counter() hook,
+ * if there is no useable hardware frame counter available.
+ */
+static u32 drm_vblank_no_hw_counter(struct drm_device *dev, unsigned int pipe)
+{
+ WARN_ON_ONCE(dev->max_vblank_count != 0);
+ return 0;
+}
+
+static u32 __get_vblank_counter(struct drm_device *dev, unsigned int pipe)
+{
+ if (drm_core_check_feature(dev, DRIVER_MODESET)) {
+ struct drm_crtc *crtc = drm_crtc_from_index(dev, pipe);
+
+ if (crtc->funcs->get_vblank_counter)
+ return crtc->funcs->get_vblank_counter(crtc);
+ }
+
+ if (dev->driver->get_vblank_counter)
+ return dev->driver->get_vblank_counter(dev, pipe);
+
+ return drm_vblank_no_hw_counter(dev, pipe);
+}
+
/*
* Reset the stored timestamp for the current vblank count to correspond
* to the last vblank occurred.
* when drm_vblank_enable() applies the diff
*/
do {
- cur_vblank = dev->driver->get_vblank_counter(dev, pipe);
+ cur_vblank = __get_vblank_counter(dev, pipe);
rc = drm_get_last_vbltimestamp(dev, pipe, &t_vblank, 0);
- } while (cur_vblank != dev->driver->get_vblank_counter(dev, pipe) && --count > 0);
+ } while (cur_vblank != __get_vblank_counter(dev, pipe) && --count > 0);
/*
* Only reinitialize corresponding vblank timestamp if high-precision query
* corresponding vblank timestamp.
*/
do {
- cur_vblank = dev->driver->get_vblank_counter(dev, pipe);
+ cur_vblank = __get_vblank_counter(dev, pipe);
rc = drm_get_last_vbltimestamp(dev, pipe, &t_vblank, flags);
- } while (cur_vblank != dev->driver->get_vblank_counter(dev, pipe) && --count > 0);
+ } while (cur_vblank != __get_vblank_counter(dev, pipe) && --count > 0);
if (dev->max_vblank_count != 0) {
/* trust the hw counter when it's around */
}
EXPORT_SYMBOL(drm_accurate_vblank_count);
+static void __disable_vblank(struct drm_device *dev, unsigned int pipe)
+{
+ if (drm_core_check_feature(dev, DRIVER_MODESET)) {
+ struct drm_crtc *crtc = drm_crtc_from_index(dev, pipe);
+
+ if (crtc->funcs->disable_vblank) {
+ crtc->funcs->disable_vblank(crtc);
+ return;
+ }
+ }
+
+ dev->driver->disable_vblank(dev, pipe);
+}
+
/*
* Disable vblank irq's on crtc, make sure that last vblank count
* of hardware and corresponding consistent software vblank counter
* hardware potentially runtime suspended.
*/
if (vblank->enabled) {
- dev->driver->disable_vblank(dev, pipe);
+ __disable_vblank(dev, pipe);
vblank->enabled = false;
}
e->event.tv_sec = now->tv_sec;
e->event.tv_usec = now->tv_usec;
- trace_drm_vblank_event_delivered(e->base.pid, e->pipe,
+ trace_drm_vblank_event_delivered(e->base.file_priv, e->pipe,
e->event.sequence);
drm_send_event_locked(dev, &e->base);
}
EXPORT_SYMBOL(drm_crtc_send_vblank_event);
+static int __enable_vblank(struct drm_device *dev, unsigned int pipe)
+{
+ if (drm_core_check_feature(dev, DRIVER_MODESET)) {
+ struct drm_crtc *crtc = drm_crtc_from_index(dev, pipe);
+
+ if (crtc->funcs->enable_vblank)
+ return crtc->funcs->enable_vblank(crtc);
+ }
+
+ return dev->driver->enable_vblank(dev, pipe);
+}
+
/**
* drm_vblank_enable - enable the vblank interrupt on a CRTC
* @dev: DRM device
* timestamps. Filtercode in drm_handle_vblank() will
* prevent double-accounting of same vblank interval.
*/
- ret = dev->driver->enable_vblank(dev, pipe);
+ ret = __enable_vblank(dev, pipe);
DRM_DEBUG("enabling vblank on crtc %u, ret: %d\n", pipe, ret);
if (ret)
atomic_dec(&vblank->refcount);
if (atomic_dec_and_test(&vblank->refcount)) {
if (drm_vblank_offdelay == 0)
return;
- else if (dev->vblank_disable_immediate || drm_vblank_offdelay < 0)
+ else if (drm_vblank_offdelay < 0)
vblank_disable_fn((unsigned long)vblank);
- else
+ else if (!dev->vblank_disable_immediate)
mod_timer(&vblank->disable_timer,
jiffies + ((drm_vblank_offdelay * HZ)/1000));
}
}
e->pipe = pipe;
- e->base.pid = current->pid;
e->event.base.type = DRM_EVENT_VBLANK;
e->event.base.length = sizeof(e->event);
e->event.user_data = vblwait->request.signal;
DRM_DEBUG("event on vblank count %u, current %u, crtc %u\n",
vblwait->request.sequence, seq, pipe);
- trace_drm_vblank_event_queued(current->pid, pipe,
+ trace_drm_vblank_event_queued(file_priv, pipe,
vblwait->request.sequence);
e->event.sequence = vblwait->request.sequence;
ret = drm_vblank_get(dev, pipe);
if (ret) {
- DRM_DEBUG("failed to acquire vblank counter, %d\n", ret);
+ DRM_DEBUG("crtc %d failed to acquire vblank counter, %d\n", pipe, ret);
return ret;
}
seq = drm_vblank_count(dev, pipe);
return drm_queue_vblank_event(dev, pipe, vblwait, file_priv);
}
- DRM_DEBUG("waiting on vblank count %u, crtc %u\n",
- vblwait->request.sequence, pipe);
- DRM_WAIT_ON(ret, vblank->queue, 3 * HZ,
- (((drm_vblank_count(dev, pipe) -
- vblwait->request.sequence) <= (1 << 23)) ||
- !vblank->enabled ||
- !dev->irq_enabled));
+ if (vblwait->request.sequence != seq) {
+ DRM_DEBUG("waiting on vblank count %u, crtc %u\n",
+ vblwait->request.sequence, pipe);
+ DRM_WAIT_ON(ret, vblank->queue, 3 * HZ,
+ (((drm_vblank_count(dev, pipe) -
+ vblwait->request.sequence) <= (1 << 23)) ||
+ !vblank->enabled ||
+ !dev->irq_enabled));
+ }
if (ret != -EINTR) {
struct timeval now;
vblwait->reply.tval_sec = now.tv_sec;
vblwait->reply.tval_usec = now.tv_usec;
- DRM_DEBUG("returning %u to client\n",
- vblwait->reply.sequence);
+ DRM_DEBUG("crtc %d returning %u to client\n",
+ pipe, vblwait->reply.sequence);
} else {
- DRM_DEBUG("vblank wait interrupted by signal\n");
+ DRM_DEBUG("crtc %d vblank wait interrupted by signal\n", pipe);
}
done:
wake_up(&vblank->queue);
drm_handle_vblank_events(dev, pipe);
+ /* With instant-off, we defer disabling the interrupt until after
+ * we finish processing the following vblank. The disable has to
+ * be last (after drm_handle_vblank_events) so that the timestamp
+ * is always accurate.
+ */
+ if (dev->vblank_disable_immediate &&
+ drm_vblank_offdelay > 0 &&
+ !atomic_read(&vblank->refcount))
+ vblank_disable_fn((unsigned long)vblank);
+
spin_unlock_irqrestore(&dev->event_lock, irqflags);
return true;
return drm_handle_vblank(crtc->dev, drm_crtc_index(crtc));
}
EXPORT_SYMBOL(drm_crtc_handle_vblank);
-
-/**
- * drm_vblank_no_hw_counter - "No hw counter" implementation of .get_vblank_counter()
- * @dev: DRM device
- * @pipe: CRTC for which to read the counter
- *
- * Drivers can plug this into the .get_vblank_counter() function if
- * there is no useable hardware frame counter available.
- *
- * Returns:
- * 0
- */
-u32 drm_vblank_no_hw_counter(struct drm_device *dev, unsigned int pipe)
-{
- WARN_ON_ONCE(dev->max_vblank_count != 0);
- return 0;
-}
-EXPORT_SYMBOL(drm_vblank_no_hw_counter);
*
*/
-#include <drm/drmP.h>
-#include <drm/drm_fb_helper.h>
+#include <linux/module.h>
#include "drm_crtc_helper_internal.h"
__drm_mm_interval_first(const struct drm_mm *mm, u64 start, u64 last)
{
return drm_mm_interval_tree_iter_first((struct rb_root *)&mm->interval_tree,
- start, last);
+ start, last) ?: (struct drm_mm_node *)&mm->head_node;
}
EXPORT_SYMBOL(__drm_mm_interval_first);
}
card_res->count_encoders = count;
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
count = 0;
connector_id = u64_to_user_ptr(card_res->connector_id_ptr);
drm_for_each_connector_iter(connector, &conn_iter) {
if (count < card_res->count_connectors &&
put_user(connector->base.id, connector_id + count)) {
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
return -EFAULT;
}
count++;
}
card_res->count_connectors = count;
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
return ret;
}
if (encoder->funcs->reset)
encoder->funcs->reset(encoder);
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter)
if (connector->funcs->reset)
connector->funcs->reset(connector);
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
}
EXPORT_SYMBOL(drm_mode_config_reset);
encoder->funcs->destroy(encoder);
}
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
/* drm_connector_list_iter holds an full reference to the
* current connector itself, which means it is inherently safe
* against unreferencing the current connector - but not against
* deleting it right away. */
- drm_connector_unreference(connector);
+ drm_connector_put(connector);
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
if (WARN_ON(!list_empty(&dev->mode_config.connector_list))) {
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter)
DRM_ERROR("connector %s leaked!\n", connector->name);
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
}
list_for_each_entry_safe(property, pt, &dev->mode_config.property_list,
list_for_each_entry_safe(blob, bt, &dev->mode_config.property_blob_list,
head_global) {
- drm_property_unreference_blob(blob);
+ drm_property_blob_put(blob);
}
/*
* Internal function to assign a slot in the object idr and optionally
* register the object into the idr.
*/
-int drm_mode_object_get_reg(struct drm_device *dev,
- struct drm_mode_object *obj,
- uint32_t obj_type,
- bool register_obj,
- void (*obj_free_cb)(struct kref *kref))
+int __drm_mode_object_add(struct drm_device *dev, struct drm_mode_object *obj,
+ uint32_t obj_type, bool register_obj,
+ void (*obj_free_cb)(struct kref *kref))
{
int ret;
}
/**
- * drm_mode_object_get - allocate a new modeset identifier
+ * drm_mode_object_add - allocate a new modeset identifier
* @dev: DRM device
* @obj: object pointer, used to generate unique ID
* @obj_type: object type
*
* Create a unique identifier based on @ptr in @dev's identifier space. Used
- * for tracking modes, CRTCs and connectors. Note that despite the _get postfix
- * modeset identifiers are _not_ reference counted. Hence don't use this for
- * reference counted modeset objects like framebuffers.
+ * for tracking modes, CRTCs and connectors.
*
* Returns:
* Zero on success, error code on failure.
*/
-int drm_mode_object_get(struct drm_device *dev,
+int drm_mode_object_add(struct drm_device *dev,
struct drm_mode_object *obj, uint32_t obj_type)
{
- return drm_mode_object_get_reg(dev, obj, obj_type, true, NULL);
+ return __drm_mode_object_add(dev, obj, obj_type, true, NULL);
}
void drm_mode_object_register(struct drm_device *dev,
*
* This function is used to look up a modeset object. It will acquire a
* reference for reference counted objects. This reference must be dropped again
- * by callind drm_mode_object_unreference().
+ * by callind drm_mode_object_put().
*/
struct drm_mode_object *drm_mode_object_find(struct drm_device *dev,
uint32_t id, uint32_t type)
EXPORT_SYMBOL(drm_mode_object_find);
/**
- * drm_mode_object_unreference - decr the object refcnt
- * @obj: mode_object
+ * drm_mode_object_put - release a mode object reference
+ * @obj: DRM mode object
*
* This function decrements the object's refcount if it is a refcounted modeset
* object. It is a no-op on any other object. This is used to drop references
- * acquired with drm_mode_object_reference().
+ * acquired with drm_mode_object_get().
*/
-void drm_mode_object_unreference(struct drm_mode_object *obj)
+void drm_mode_object_put(struct drm_mode_object *obj)
{
if (obj->free_cb) {
DRM_DEBUG("OBJ ID: %d (%d)\n", obj->id, kref_read(&obj->refcount));
kref_put(&obj->refcount, obj->free_cb);
}
}
-EXPORT_SYMBOL(drm_mode_object_unreference);
+EXPORT_SYMBOL(drm_mode_object_put);
/**
- * drm_mode_object_reference - incr the object refcnt
- * @obj: mode_object
+ * drm_mode_object_get - acquire a mode object reference
+ * @obj: DRM mode object
*
* This function increments the object's refcount if it is a refcounted modeset
* object. It is a no-op on any other object. References should be dropped again
- * by calling drm_mode_object_unreference().
+ * by calling drm_mode_object_put().
*/
-void drm_mode_object_reference(struct drm_mode_object *obj)
+void drm_mode_object_get(struct drm_mode_object *obj)
{
if (obj->free_cb) {
DRM_DEBUG("OBJ ID: %d (%d)\n", obj->id, kref_read(&obj->refcount));
kref_get(&obj->refcount);
}
}
-EXPORT_SYMBOL(drm_mode_object_reference);
+EXPORT_SYMBOL(drm_mode_object_get);
/**
* drm_object_attach_property - attach a property to a modeset object
&arg->count_props);
out_unref:
- drm_mode_object_unreference(obj);
+ drm_mode_object_put(obj);
out:
drm_modeset_unlock_all(dev);
return ret;
drm_property_change_valid_put(property, ref);
out_unref:
- drm_mode_object_unreference(arg_obj);
+ drm_mode_object_put(arg_obj);
out:
drm_modeset_unlock_all(dev);
return ret;
if (!nmode)
return NULL;
- if (drm_mode_object_get(dev, &nmode->base, DRM_MODE_OBJECT_MODE)) {
+ if (drm_mode_object_add(dev, &nmode->base, DRM_MODE_OBJECT_MODE)) {
kfree(nmode);
return NULL;
}
#include <linux/slab.h>
#include <linux/dma-mapping.h>
#include <linux/export.h>
+#include <drm/drm_pci.h>
#include <drm/drmP.h>
#include "drm_internal.h"
#include "drm_legacy.h"
* @size: size of block to allocate
* @align: alignment of block
*
+ * FIXME: This is a needless abstraction of the Linux dma-api and should be
+ * removed.
+ *
* Return: A handle to the allocated memory block on success or NULL on
* failure.
*/
* drm_pci_free - Free a PCI consistent memory block
* @dev: DRM device
* @dmah: handle to memory block
+ *
+ * FIXME: This is a needless abstraction of the Linux dma-api and should be
+ * removed.
*/
void drm_pci_free(struct drm_device * dev, drm_dma_handle_t * dmah)
{
struct drm_mode_config *config = &dev->mode_config;
int ret;
- ret = drm_mode_object_get(dev, &plane->base, DRM_MODE_OBJECT_PLANE);
+ ret = drm_mode_object_add(dev, &plane->base, DRM_MODE_OBJECT_PLANE);
if (ret)
return ret;
return;
}
/* disconnect the plane from the fb and crtc: */
- drm_framebuffer_unreference(plane->old_fb);
+ drm_framebuffer_put(plane->old_fb);
plane->old_fb = NULL;
plane->fb = NULL;
plane->crtc = NULL;
out:
if (fb)
- drm_framebuffer_unreference(fb);
+ drm_framebuffer_put(fb);
if (plane->old_fb)
- drm_framebuffer_unreference(plane->old_fb);
+ drm_framebuffer_put(plane->old_fb);
plane->old_fb = NULL;
return ret;
} else {
fb = crtc->cursor->fb;
if (fb)
- drm_framebuffer_reference(fb);
+ drm_framebuffer_get(fb);
}
if (req->flags & DRM_MODE_CURSOR_MOVE) {
if (ret && crtc->funcs->page_flip_target)
drm_crtc_vblank_put(crtc);
if (fb)
- drm_framebuffer_unreference(fb);
+ drm_framebuffer_put(fb);
if (crtc->primary->old_fb)
- drm_framebuffer_unreference(crtc->primary->old_fb);
+ drm_framebuffer_put(crtc->primary->old_fb);
crtc->primary->old_fb = NULL;
drm_modeset_unlock_crtc(crtc);
*/
WARN_ON(!drm_modeset_is_locked(&dev->mode_config.connection_mutex));
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->encoder && connector->encoder->crtc == crtc) {
if (connector_list != NULL && count < num_connectors)
count++;
}
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
return count;
}
goto out;
}
- if (plane_funcs->prepare_fb && plane_state->fb &&
- plane_state->fb != old_fb) {
+ if (plane_funcs->prepare_fb && plane_state->fb != old_fb) {
ret = plane_funcs->prepare_fb(plane,
plane_state);
if (ret)
* Drivers may optionally implement the ->atomic_disable callback, so
* special-case that here.
*/
- if (drm_atomic_plane_disabling(plane, plane_state) &&
+ if (drm_atomic_plane_disabling(plane_state, plane->state) &&
plane_funcs->atomic_disable)
plane_funcs->atomic_disable(plane, plane_state);
else
+++ /dev/null
-/*
- * Derived from drm_pci.c
- *
- * Copyright 2003 José Fonseca.
- * Copyright 2003 Leif Delgass.
- * Copyright (c) 2009, Code Aurora Forum.
- * All Rights Reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice (including the next
- * paragraph) shall be included in all copies or substantial portions of the
- * Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
- * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- */
-
-#include <linux/export.h>
-#include <drm/drmP.h>
-
-/*
- * Register.
- *
- * \param platdev - Platform device struture
- * \return zero on success or a negative number on failure.
- *
- * Attempt to gets inter module "drm" information. If we are first
- * then register the character device and inter module information.
- * Try and register, if we fail to register, backout previous work.
- */
-
-static int drm_get_platform_dev(struct platform_device *platdev,
- struct drm_driver *driver)
-{
- struct drm_device *dev;
- int ret;
-
- DRM_DEBUG("\n");
-
- dev = drm_dev_alloc(driver, &platdev->dev);
- if (IS_ERR(dev))
- return PTR_ERR(dev);
-
- dev->platformdev = platdev;
-
- ret = drm_dev_register(dev, 0);
- if (ret)
- goto err_free;
-
- return 0;
-
-err_free:
- drm_dev_unref(dev);
- return ret;
-}
-
-/**
- * drm_platform_init - Register a platform device with the DRM subsystem
- * @driver: DRM device driver
- * @platform_device: platform device to register
- *
- * Registers the specified DRM device driver and platform device with the DRM
- * subsystem, initializing a drm_device structure and calling the driver's
- * .load() function.
- *
- * NOTE: This function is deprecated, please use drm_dev_alloc() and
- * drm_dev_register() instead and remove your &drm_driver.load callback.
- *
- * Return: 0 on success or a negative error code on failure.
- */
-int drm_platform_init(struct drm_driver *driver, struct platform_device *platform_device)
-{
- DRM_DEBUG("\n");
-
- return drm_get_platform_dev(platform_device, driver);
-}
-EXPORT_SYMBOL(drm_platform_init);
#include <linux/export.h>
#include <linux/dma-buf.h>
#include <linux/rbtree.h>
-#include <drm/drmP.h>
+#include <drm/drm_prime.h>
#include <drm/drm_gem.h>
+#include <drm/drmP.h>
#include "drm_internal.h"
return dma_buf;
drm_dev_ref(dev);
- drm_gem_object_reference(exp_info->priv);
+ drm_gem_object_get(exp_info->priv);
return dma_buf;
}
struct drm_device *dev = obj->dev;
/* drop the reference on the export fd holds */
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
drm_dev_unref(dev);
}
fail_put_dmabuf:
dma_buf_put(dmabuf);
out:
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
out_unlock:
mutex_unlock(&file_priv->prime.lock);
* Importing dmabuf exported from out own gem increases
* refcount on gem itself instead of f_count of dmabuf.
*/
- drm_gem_object_reference(obj);
+ drm_gem_object_get(obj);
return obj;
}
}
/* _handle_create_tail unconditionally unlocks dev->object_name_lock. */
ret = drm_gem_handle_create_tail(file_priv, obj, handle);
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_put_unlocked(obj);
if (ret)
goto out_put;
void __drm_printfn_info(struct drm_printer *p, struct va_format *vaf)
{
- dev_printk(KERN_INFO, p->arg, "[" DRM_NAME "] %pV", vaf);
+ dev_info(p->arg, "[" DRM_NAME "] %pV", vaf);
}
EXPORT_SYMBOL(__drm_printfn_info);
if (!dev->mode_config.poll_enabled || !drm_kms_helper_poll)
return;
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->polled & (DRM_CONNECTOR_POLL_CONNECT |
DRM_CONNECTOR_POLL_DISCONNECT))
poll = true;
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
if (dev->mode_config.delayed_event) {
/*
count = drm_add_edid_modes(connector, edid);
drm_edid_to_eld(connector, edid);
} else {
- count = drm_load_edid_firmware(connector);
+ struct edid *edid = drm_load_edid_firmware(connector);
+ if (!IS_ERR_OR_NULL(edid)) {
+ drm_mode_connector_update_edid_property(connector, edid);
+ count = drm_add_edid_modes(connector, edid);
+ drm_edid_to_eld(connector, edid);
+ kfree(edid);
+ }
if (count == 0)
count = (*connector_funcs->get_modes)(connector);
}
goto out;
}
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
/* Ignore forced connectors. */
if (connector->force)
changed = true;
}
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
mutex_unlock(&dev->mode_config.mutex);
return false;
mutex_lock(&dev->mode_config.mutex);
- drm_connector_list_iter_get(dev, &conn_iter);
+ drm_connector_list_iter_begin(dev, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
/* Only handle HPD capable connectors. */
if (!(connector->polled & DRM_CONNECTOR_POLL_HPD))
if (old_status != connector->status)
changed = true;
}
- drm_connector_list_iter_put(&conn_iter);
+ drm_connector_list_iter_end(&conn_iter);
mutex_unlock(&dev->mode_config.mutex);
if (changed)
goto fail;
}
- ret = drm_mode_object_get(dev, &property->base, DRM_MODE_OBJECT_PROPERTY);
+ ret = drm_mode_object_add(dev, &property->base, DRM_MODE_OBJECT_PROPERTY);
if (ret)
goto fail;
if (data)
memcpy(blob->data, data, length);
- ret = drm_mode_object_get_reg(dev, &blob->base, DRM_MODE_OBJECT_BLOB,
- true, drm_property_free_blob);
+ ret = __drm_mode_object_add(dev, &blob->base, DRM_MODE_OBJECT_BLOB,
+ true, drm_property_free_blob);
if (ret) {
kfree(blob);
return ERR_PTR(-EINVAL);
EXPORT_SYMBOL(drm_property_create_blob);
/**
- * drm_property_unreference_blob - Unreference a blob property
- * @blob: Pointer to blob property
+ * drm_property_blob_put - release a blob property reference
+ * @blob: DRM blob property
*
- * Drop a reference on a blob property. May free the object.
+ * Releases a reference to a blob property. May free the object.
*/
-void drm_property_unreference_blob(struct drm_property_blob *blob)
+void drm_property_blob_put(struct drm_property_blob *blob)
{
if (!blob)
return;
- drm_mode_object_unreference(&blob->base);
+ drm_mode_object_put(&blob->base);
}
-EXPORT_SYMBOL(drm_property_unreference_blob);
+EXPORT_SYMBOL(drm_property_blob_put);
void drm_property_destroy_user_blobs(struct drm_device *dev,
struct drm_file *file_priv)
*/
list_for_each_entry_safe(blob, bt, &file_priv->blobs, head_file) {
list_del_init(&blob->head_file);
- drm_property_unreference_blob(blob);
+ drm_property_blob_put(blob);
}
}
/**
- * drm_property_reference_blob - Take a reference on an existing property
- * @blob: Pointer to blob property
+ * drm_property_blob_get - acquire blob property reference
+ * @blob: DRM blob property
*
- * Take a new reference on an existing blob property. Returns @blob, which
+ * Acquires a reference to an existing blob property. Returns @blob, which
* allows this to be used as a shorthand in assignments.
*/
-struct drm_property_blob *drm_property_reference_blob(struct drm_property_blob *blob)
+struct drm_property_blob *drm_property_blob_get(struct drm_property_blob *blob)
{
- drm_mode_object_reference(&blob->base);
+ drm_mode_object_get(&blob->base);
return blob;
}
-EXPORT_SYMBOL(drm_property_reference_blob);
+EXPORT_SYMBOL(drm_property_blob_get);
/**
* drm_property_lookup_blob - look up a blob property and take a reference
*
* If successful, this takes an additional reference to the blob property.
* callers need to make sure to eventually unreference the returned property
- * again, using @drm_property_unreference_blob.
+ * again, using drm_property_blob_put().
*
* Return:
* NULL on failure, pointer to the blob on success.
goto err_created;
}
- drm_property_unreference_blob(old_blob);
+ drm_property_blob_put(old_blob);
*replace = new_blob;
return 0;
err_created:
- drm_property_unreference_blob(new_blob);
+ drm_property_blob_put(new_blob);
return ret;
}
EXPORT_SYMBOL(drm_property_replace_global_blob);
}
out_resp->length = blob->length;
unref:
- drm_property_unreference_blob(blob);
+ drm_property_blob_put(blob);
return ret;
}
return 0;
out_blob:
- drm_property_unreference_blob(blob);
+ drm_property_blob_put(blob);
return ret;
}
mutex_unlock(&dev->mode_config.blob_lock);
/* One reference from lookup, and one from the filp. */
- drm_property_unreference_blob(blob);
- drm_property_unreference_blob(blob);
+ drm_property_blob_put(blob);
+ drm_property_blob_put(blob);
return 0;
err:
mutex_unlock(&dev->mode_config.blob_lock);
- drm_property_unreference_blob(blob);
+ drm_property_blob_put(blob);
return ret;
}
return;
if (drm_property_type_is(property, DRM_MODE_PROP_OBJECT)) {
- drm_mode_object_unreference(ref);
+ drm_mode_object_put(ref);
} else if (drm_property_type_is(property, DRM_MODE_PROP_BLOB))
- drm_property_unreference_blob(obj_to_blob(ref));
+ drm_property_blob_put(obj_to_blob(ref));
}
--- /dev/null
+/*
+ * Copyright (c) 2015 NVIDIA Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sub license,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the
+ * next paragraph) shall be included in all copies or substantial portions
+ * of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include <linux/slab.h>
+#include <linux/delay.h>
+
+#include <drm/drm_scdc_helper.h>
+#include <drm/drmP.h>
+
+/**
+ * DOC: scdc helpers
+ *
+ * Status and Control Data Channel (SCDC) is a mechanism introduced by the
+ * HDMI 2.0 specification. It is a point-to-point protocol that allows the
+ * HDMI source and HDMI sink to exchange data. The same I2C interface that
+ * is used to access EDID serves as the transport mechanism for SCDC.
+ */
+
+#define SCDC_I2C_SLAVE_ADDRESS 0x54
+
+/**
+ * drm_scdc_read - read a block of data from SCDC
+ * @adapter: I2C controller
+ * @offset: start offset of block to read
+ * @buffer: return location for the block to read
+ * @size: size of the block to read
+ *
+ * Reads a block of data from SCDC, starting at a given offset.
+ *
+ * Returns:
+ * 0 on success, negative error code on failure.
+ */
+ssize_t drm_scdc_read(struct i2c_adapter *adapter, u8 offset, void *buffer,
+ size_t size)
+{
+ int ret;
+ struct i2c_msg msgs[2] = {
+ {
+ .addr = SCDC_I2C_SLAVE_ADDRESS,
+ .flags = 0,
+ .len = 1,
+ .buf = &offset,
+ }, {
+ .addr = SCDC_I2C_SLAVE_ADDRESS,
+ .flags = I2C_M_RD,
+ .len = size,
+ .buf = buffer,
+ }
+ };
+
+ ret = i2c_transfer(adapter, msgs, ARRAY_SIZE(msgs));
+ if (ret < 0)
+ return ret;
+ if (ret != ARRAY_SIZE(msgs))
+ return -EPROTO;
+
+ return 0;
+}
+EXPORT_SYMBOL(drm_scdc_read);
+
+/**
+ * drm_scdc_write - write a block of data to SCDC
+ * @adapter: I2C controller
+ * @offset: start offset of block to write
+ * @buffer: block of data to write
+ * @size: size of the block to write
+ *
+ * Writes a block of data to SCDC, starting at a given offset.
+ *
+ * Returns:
+ * 0 on success, negative error code on failure.
+ */
+ssize_t drm_scdc_write(struct i2c_adapter *adapter, u8 offset,
+ const void *buffer, size_t size)
+{
+ struct i2c_msg msg = {
+ .addr = SCDC_I2C_SLAVE_ADDRESS,
+ .flags = 0,
+ .len = 1 + size,
+ .buf = NULL,
+ };
+ void *data;
+ int err;
+
+ data = kmalloc(1 + size, GFP_TEMPORARY);
+ if (!data)
+ return -ENOMEM;
+
+ msg.buf = data;
+
+ memcpy(data, &offset, sizeof(offset));
+ memcpy(data + 1, buffer, size);
+
+ err = i2c_transfer(adapter, &msg, 1);
+
+ kfree(data);
+
+ if (err < 0)
+ return err;
+ if (err != 1)
+ return -EPROTO;
+
+ return 0;
+}
+EXPORT_SYMBOL(drm_scdc_write);
+
+/**
+ * drm_scdc_check_scrambling_status - what is status of scrambling?
+ * @adapter: I2C adapter for DDC channel
+ *
+ * Reads the scrambler status over SCDC, and checks the
+ * scrambling status.
+ *
+ * Returns:
+ * True if the scrambling is enabled, false otherwise.
+ */
+
+bool drm_scdc_get_scrambling_status(struct i2c_adapter *adapter)
+{
+ u8 status;
+ int ret;
+
+ ret = drm_scdc_readb(adapter, SCDC_SCRAMBLER_STATUS, &status);
+ if (ret < 0) {
+ DRM_ERROR("Failed to read scrambling status, error %d\n", ret);
+ return false;
+ }
+
+ return status & SCDC_SCRAMBLING_STATUS;
+}
+EXPORT_SYMBOL(drm_scdc_get_scrambling_status);
+
+/**
+ * drm_scdc_set_scrambling - enable scrambling
+ * @adapter: I2C adapter for DDC channel
+ * @enable: bool to indicate if scrambling is to be enabled/disabled
+ *
+ * Writes the TMDS config register over SCDC channel, and:
+ * enables scrambling when enable = 1
+ * disables scrambling when enable = 0
+ *
+ * Returns:
+ * True if scrambling is set/reset successfully, false otherwise.
+ */
+
+bool drm_scdc_set_scrambling(struct i2c_adapter *adapter, bool enable)
+{
+ u8 config;
+ int ret;
+
+ ret = drm_scdc_readb(adapter, SCDC_TMDS_CONFIG, &config);
+ if (ret < 0) {
+ DRM_ERROR("Failed to read tmds config, err=%d\n", ret);
+ return false;
+ }
+
+ if (enable)
+ config |= SCDC_SCRAMBLING_ENABLE;
+ else
+ config &= ~SCDC_SCRAMBLING_ENABLE;
+
+ ret = drm_scdc_writeb(adapter, SCDC_TMDS_CONFIG, config);
+ if (ret < 0) {
+ DRM_ERROR("Failed to enable scrambling, error %d\n", ret);
+ return false;
+ }
+
+ return true;
+}
+EXPORT_SYMBOL(drm_scdc_set_scrambling);
+
+/**
+ * drm_scdc_set_high_tmds_clock_ratio - set TMDS clock ratio
+ * @adapter: I2C adapter for DDC channel
+ * @set: ret or reset the high clock ratio
+ *
+ * TMDS clock ratio calculations go like this:
+ * TMDS character = 10 bit TMDS encoded value
+ * TMDS character rate = The rate at which TMDS characters are transmitted(Mcsc)
+ * TMDS bit rate = 10x TMDS character rate
+ * As per the spec:
+ * TMDS clock rate for pixel clock < 340 MHz = 1x the character rate
+ * = 1/10 pixel clock rate
+ * TMDS clock rate for pixel clock > 340 MHz = 0.25x the character rate
+ * = 1/40 pixel clock rate
+ *
+ * Writes to the TMDS config register over SCDC channel, and:
+ * sets TMDS clock ratio to 1/40 when set = 1
+ * sets TMDS clock ratio to 1/10 when set = 0
+ *
+ * Returns:
+ * True if write is successful, false otherwise.
+ */
+bool drm_scdc_set_high_tmds_clock_ratio(struct i2c_adapter *adapter, bool set)
+{
+ u8 config;
+ int ret;
+
+ ret = drm_scdc_readb(adapter, SCDC_TMDS_CONFIG, &config);
+ if (ret < 0) {
+ DRM_ERROR("Failed to read tmds config, err=%d\n", ret);
+ return false;
+ }
+
+ if (set)
+ config |= SCDC_TMDS_BIT_CLOCK_RATIO_BY_40;
+ else
+ config &= ~SCDC_TMDS_BIT_CLOCK_RATIO_BY_40;
+
+ ret = drm_scdc_writeb(adapter, SCDC_TMDS_CONFIG, config);
+ if (ret < 0) {
+ DRM_ERROR("Failed to set TMDS clock ratio, error %d\n", ret);
+ return false;
+ }
+
+ /*
+ * The spec says that a source should wait minimum 1ms and maximum
+ * 100ms after writing the TMDS config for clock ratio. Lets allow a
+ * wait of upto 2ms here.
+ */
+ usleep_range(1000, 2000);
+ return true;
+}
+EXPORT_SYMBOL(drm_scdc_set_high_tmds_clock_ratio);
int ret;
pipe = container_of(plane, struct drm_simple_display_pipe, plane);
- crtc_state = drm_atomic_get_existing_crtc_state(plane_state->state,
- &pipe->crtc);
+ crtc_state = drm_atomic_get_new_crtc_state(plane_state->state,
+ &pipe->crtc);
if (crtc_state->enable != !!plane_state->crtc)
return -EINVAL; /* plane must match crtc enable state */
);
TRACE_EVENT(drm_vblank_event_queued,
- TP_PROTO(pid_t pid, int crtc, unsigned int seq),
- TP_ARGS(pid, crtc, seq),
+ TP_PROTO(struct drm_file *file, int crtc, unsigned int seq),
+ TP_ARGS(file, crtc, seq),
TP_STRUCT__entry(
- __field(pid_t, pid)
+ __field(struct drm_file *, file)
__field(int, crtc)
__field(unsigned int, seq)
),
TP_fast_assign(
- __entry->pid = pid;
+ __entry->file = file;
__entry->crtc = crtc;
__entry->seq = seq;
),
- TP_printk("pid=%d, crtc=%d, seq=%u", __entry->pid, __entry->crtc, \
+ TP_printk("file=%p, crtc=%d, seq=%u", __entry->file, __entry->crtc, \
__entry->seq)
);
TRACE_EVENT(drm_vblank_event_delivered,
- TP_PROTO(pid_t pid, int crtc, unsigned int seq),
- TP_ARGS(pid, crtc, seq),
+ TP_PROTO(struct drm_file *file, int crtc, unsigned int seq),
+ TP_ARGS(file, crtc, seq),
TP_STRUCT__entry(
- __field(pid_t, pid)
+ __field(struct drm_file *, file)
__field(int, crtc)
__field(unsigned int, seq)
),
TP_fast_assign(
- __entry->pid = pid;
+ __entry->file = file;
__entry->crtc = crtc;
__entry->seq = seq;
),
- TP_printk("pid=%d, crtc=%d, seq=%u", __entry->pid, __entry->crtc, \
+ TP_printk("file=%p, crtc=%d, seq=%u", __entry->file, __entry->crtc, \
__entry->seq)
);
struct exynos_dp_device *dp = to_dp(plat_data);
int ret;
- drm_connector_register(connector);
dp->connector = connector;
/* Pre-empt DP connector creation if there's a bridge */
kfree(exynos_crtc);
}
+static int exynos_drm_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ struct exynos_drm_crtc *exynos_crtc = to_exynos_crtc(crtc);
+
+ if (exynos_crtc->ops->enable_vblank)
+ return exynos_crtc->ops->enable_vblank(exynos_crtc);
+
+ return 0;
+}
+
+static void exynos_drm_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+ struct exynos_drm_crtc *exynos_crtc = to_exynos_crtc(crtc);
+
+ if (exynos_crtc->ops->disable_vblank)
+ exynos_crtc->ops->disable_vblank(exynos_crtc);
+}
+
static const struct drm_crtc_funcs exynos_crtc_funcs = {
.set_config = drm_atomic_helper_set_config,
.page_flip = drm_atomic_helper_page_flip,
.reset = drm_atomic_helper_crtc_reset,
.atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .enable_vblank = exynos_drm_crtc_enable_vblank,
+ .disable_vblank = exynos_drm_crtc_disable_vblank,
};
struct exynos_drm_crtc *exynos_drm_crtc_create(struct drm_device *drm_dev,
return ERR_PTR(ret);
}
-int exynos_drm_crtc_enable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- struct exynos_drm_crtc *exynos_crtc = exynos_drm_crtc_from_pipe(dev,
- pipe);
-
- if (exynos_crtc->ops->enable_vblank)
- return exynos_crtc->ops->enable_vblank(exynos_crtc);
-
- return 0;
-}
-
-void exynos_drm_crtc_disable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- struct exynos_drm_crtc *exynos_crtc = exynos_drm_crtc_from_pipe(dev,
- pipe);
-
- if (exynos_crtc->ops->disable_vblank)
- exynos_crtc->ops->disable_vblank(exynos_crtc);
-}
-
int exynos_drm_crtc_get_pipe_from_type(struct drm_device *drm_dev,
enum exynos_drm_output_type out_type)
{
enum exynos_drm_output_type type,
const struct exynos_drm_crtc_ops *ops,
void *context);
-int exynos_drm_crtc_enable_vblank(struct drm_device *dev, unsigned int pipe);
-void exynos_drm_crtc_disable_vblank(struct drm_device *dev, unsigned int pipe);
void exynos_drm_crtc_wait_pending_update(struct exynos_drm_crtc *exynos_crtc);
void exynos_drm_crtc_finish_update(struct exynos_drm_crtc *exynos_crtc,
struct exynos_drm_plane *exynos_plane);
}
drm_connector_helper_add(connector, &exynos_dpi_connector_helper_funcs);
- drm_connector_register(connector);
drm_mode_connector_attach_encoder(connector, encoder);
return 0;
#include <drm/exynos_drm.h>
#include "exynos_drm_drv.h"
-#include "exynos_drm_crtc.h"
#include "exynos_drm_fbdev.h"
#include "exynos_drm_fb.h"
#include "exynos_drm_gem.h"
static struct device *exynos_drm_get_dma_device(void);
-static int exynos_drm_load(struct drm_device *dev, unsigned long flags)
-{
- struct exynos_drm_private *private;
- struct drm_encoder *encoder;
- unsigned int clone_mask;
- int cnt, ret;
-
- private = kzalloc(sizeof(struct exynos_drm_private), GFP_KERNEL);
- if (!private)
- return -ENOMEM;
-
- init_waitqueue_head(&private->wait);
- spin_lock_init(&private->lock);
-
- dev_set_drvdata(dev->dev, dev);
- dev->dev_private = (void *)private;
-
- /* the first real CRTC device is used for all dma mapping operations */
- private->dma_dev = exynos_drm_get_dma_device();
- if (!private->dma_dev) {
- DRM_ERROR("no device found for DMA mapping operations.\n");
- ret = -ENODEV;
- goto err_free_private;
- }
- DRM_INFO("Exynos DRM: using %s device for DMA mapping operations\n",
- dev_name(private->dma_dev));
-
- /* create common IOMMU mapping for all devices attached to Exynos DRM */
- ret = drm_create_iommu_mapping(dev);
- if (ret < 0) {
- DRM_ERROR("failed to create iommu mapping.\n");
- goto err_free_private;
- }
-
- drm_mode_config_init(dev);
-
- exynos_drm_mode_config_init(dev);
-
- /* setup possible_clones. */
- cnt = 0;
- clone_mask = 0;
- list_for_each_entry(encoder, &dev->mode_config.encoder_list, head)
- clone_mask |= (1 << (cnt++));
-
- list_for_each_entry(encoder, &dev->mode_config.encoder_list, head)
- encoder->possible_clones = clone_mask;
-
- platform_set_drvdata(dev->platformdev, dev);
-
- /* Try to bind all sub drivers. */
- ret = component_bind_all(dev->dev, dev);
- if (ret)
- goto err_mode_config_cleanup;
-
- ret = drm_vblank_init(dev, dev->mode_config.num_crtc);
- if (ret)
- goto err_unbind_all;
-
- /* Probe non kms sub drivers and virtual display driver. */
- ret = exynos_drm_device_subdrv_probe(dev);
- if (ret)
- goto err_cleanup_vblank;
-
- drm_mode_config_reset(dev);
-
- /*
- * enable drm irq mode.
- * - with irq_enabled = true, we can use the vblank feature.
- *
- * P.S. note that we wouldn't use drm irq handler but
- * just specific driver own one instead because
- * drm framework supports only one irq handler.
- */
- dev->irq_enabled = true;
-
- /* init kms poll for handling hpd */
- drm_kms_helper_poll_init(dev);
-
- /* force connectors detection */
- drm_helper_hpd_irq_event(dev);
-
- return 0;
-
-err_cleanup_vblank:
- drm_vblank_cleanup(dev);
-err_unbind_all:
- component_unbind_all(dev->dev, dev);
-err_mode_config_cleanup:
- drm_mode_config_cleanup(dev);
- drm_release_iommu_mapping(dev);
-err_free_private:
- kfree(private);
-
- return ret;
-}
-
-static void exynos_drm_unload(struct drm_device *dev)
-{
- exynos_drm_device_subdrv_remove(dev);
-
- exynos_drm_fbdev_fini(dev);
- drm_kms_helper_poll_fini(dev);
-
- drm_vblank_cleanup(dev);
- component_unbind_all(dev->dev, dev);
- drm_mode_config_cleanup(dev);
- drm_release_iommu_mapping(dev);
-
- kfree(dev->dev_private);
- dev->dev_private = NULL;
-}
-
int exynos_atomic_check(struct drm_device *dev,
struct drm_atomic_state *state)
{
static struct drm_driver exynos_drm_driver = {
.driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_PRIME
| DRIVER_ATOMIC | DRIVER_RENDER,
- .load = exynos_drm_load,
- .unload = exynos_drm_unload,
.open = exynos_drm_open,
.preclose = exynos_drm_preclose,
.lastclose = exynos_drm_lastclose,
.postclose = exynos_drm_postclose,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = exynos_drm_crtc_enable_vblank,
- .disable_vblank = exynos_drm_crtc_disable_vblank,
.gem_free_object_unlocked = exynos_drm_gem_free_object,
.gem_vm_ops = &exynos_drm_gem_vm_ops,
.dumb_create = exynos_drm_gem_dumb_create,
static int exynos_drm_bind(struct device *dev)
{
- return drm_platform_init(&exynos_drm_driver, to_platform_device(dev));
+ struct exynos_drm_private *private;
+ struct drm_encoder *encoder;
+ struct drm_device *drm;
+ unsigned int clone_mask;
+ int cnt, ret;
+
+ drm = drm_dev_alloc(&exynos_drm_driver, dev);
+ if (IS_ERR(drm))
+ return PTR_ERR(drm);
+
+ private = kzalloc(sizeof(struct exynos_drm_private), GFP_KERNEL);
+ if (!private) {
+ ret = -ENOMEM;
+ goto err_free_drm;
+ }
+
+ init_waitqueue_head(&private->wait);
+ spin_lock_init(&private->lock);
+
+ dev_set_drvdata(dev, drm);
+ drm->dev_private = (void *)private;
+
+ /* the first real CRTC device is used for all dma mapping operations */
+ private->dma_dev = exynos_drm_get_dma_device();
+ if (!private->dma_dev) {
+ DRM_ERROR("no device found for DMA mapping operations.\n");
+ ret = -ENODEV;
+ goto err_free_private;
+ }
+ DRM_INFO("Exynos DRM: using %s device for DMA mapping operations\n",
+ dev_name(private->dma_dev));
+
+ /* create common IOMMU mapping for all devices attached to Exynos DRM */
+ ret = drm_create_iommu_mapping(drm);
+ if (ret < 0) {
+ DRM_ERROR("failed to create iommu mapping.\n");
+ goto err_free_private;
+ }
+
+ drm_mode_config_init(drm);
+
+ exynos_drm_mode_config_init(drm);
+
+ /* setup possible_clones. */
+ cnt = 0;
+ clone_mask = 0;
+ list_for_each_entry(encoder, &drm->mode_config.encoder_list, head)
+ clone_mask |= (1 << (cnt++));
+
+ list_for_each_entry(encoder, &drm->mode_config.encoder_list, head)
+ encoder->possible_clones = clone_mask;
+
+ /* Try to bind all sub drivers. */
+ ret = component_bind_all(drm->dev, drm);
+ if (ret)
+ goto err_mode_config_cleanup;
+
+ ret = drm_vblank_init(drm, drm->mode_config.num_crtc);
+ if (ret)
+ goto err_unbind_all;
+
+ /* Probe non kms sub drivers and virtual display driver. */
+ ret = exynos_drm_device_subdrv_probe(drm);
+ if (ret)
+ goto err_cleanup_vblank;
+
+ drm_mode_config_reset(drm);
+
+ /*
+ * enable drm irq mode.
+ * - with irq_enabled = true, we can use the vblank feature.
+ *
+ * P.S. note that we wouldn't use drm irq handler but
+ * just specific driver own one instead because
+ * drm framework supports only one irq handler.
+ */
+ drm->irq_enabled = true;
+
+ /* init kms poll for handling hpd */
+ drm_kms_helper_poll_init(drm);
+
+ /* force connectors detection */
+ drm_helper_hpd_irq_event(drm);
+
+ /* register the DRM device */
+ ret = drm_dev_register(drm, 0);
+ if (ret < 0)
+ goto err_cleanup_fbdev;
+
+ return 0;
+
+err_cleanup_fbdev:
+ exynos_drm_fbdev_fini(drm);
+ drm_kms_helper_poll_fini(drm);
+ exynos_drm_device_subdrv_remove(drm);
+err_cleanup_vblank:
+ drm_vblank_cleanup(drm);
+err_unbind_all:
+ component_unbind_all(drm->dev, drm);
+err_mode_config_cleanup:
+ drm_mode_config_cleanup(drm);
+ drm_release_iommu_mapping(drm);
+err_free_private:
+ kfree(private);
+err_free_drm:
+ drm_dev_unref(drm);
+
+ return ret;
}
static void exynos_drm_unbind(struct device *dev)
{
- drm_put_dev(dev_get_drvdata(dev));
+ struct drm_device *drm = dev_get_drvdata(dev);
+
+ drm_dev_unregister(drm);
+
+ exynos_drm_device_subdrv_remove(drm);
+
+ exynos_drm_fbdev_fini(drm);
+ drm_kms_helper_poll_fini(drm);
+
+ component_unbind_all(drm->dev, drm);
+ drm_mode_config_cleanup(drm);
+ drm_release_iommu_mapping(drm);
+
+ kfree(drm->dev_private);
+ drm->dev_private = NULL;
+
+ drm_dev_unref(drm);
}
static const struct component_master_ops exynos_drm_ops = {
wait_queue_head_t wait;
};
-static inline struct exynos_drm_crtc *
-exynos_drm_crtc_from_pipe(struct drm_device *dev, int pipe)
-{
- struct drm_crtc *crtc = drm_crtc_from_index(dev, pipe);
-
- return to_exynos_crtc(crtc);
-}
-
static inline struct device *to_dma_dev(struct drm_device *dev)
{
struct exynos_drm_private *priv = dev->dev_private;
}
drm_connector_helper_add(connector, &exynos_dsi_connector_helper_funcs);
- drm_connector_register(connector);
drm_mode_connector_attach_encoder(connector, encoder);
return 0;
VM_MAP, pgprot_writecombine(PAGE_KERNEL));
if (!exynos_gem->kvaddr) {
DRM_ERROR("failed to map pages to kernel space.\n");
- drm_fb_helper_release_fbi(helper);
return -EIO;
}
struct exynos_drm_gem *exynos_gem;
struct drm_device *dev = helper->dev;
struct drm_mode_fb_cmd2 mode_cmd = { 0 };
- struct platform_device *pdev = dev->platformdev;
unsigned long size;
int ret;
* memory area.
*/
if (IS_ERR(exynos_gem) && is_drm_iommu_supported(dev)) {
- dev_warn(&pdev->dev, "contiguous FB allocation failed, falling back to non-contiguous\n");
+ dev_warn(dev->dev, "contiguous FB allocation failed, falling back to non-contiguous\n");
exynos_gem = exynos_drm_gem_create(dev, EXYNOS_BO_NONCONTIG,
size);
}
}
drm_fb_helper_unregister_fbi(fb_helper);
- drm_fb_helper_release_fbi(fb_helper);
drm_fb_helper_fini(fb_helper);
}
}
drm_connector_helper_add(connector, &vidi_connector_helper_funcs);
- drm_connector_register(connector);
drm_mode_connector_attach_encoder(connector, encoder);
return 0;
#include <drm/exynos_drm.h>
-#include "exynos_drm_drv.h"
#include "exynos_drm_crtc.h"
#define HOTPLUG_DEBOUNCE_MS 1100
}
drm_connector_helper_add(connector, &hdmi_connector_helper_funcs);
- drm_connector_register(connector);
drm_mode_connector_attach_encoder(connector, encoder);
if (hdata->bridge) {
struct drm_device *drm_dev = data;
struct hdmi_context *hdata = dev_get_drvdata(dev);
struct drm_encoder *encoder = &hdata->encoder;
+ struct exynos_drm_crtc *exynos_crtc;
+ struct drm_crtc *crtc;
int ret, pipe;
hdata->drm_dev = drm_dev;
hdata->phy_clk.enable = hdmiphy_clk_enable;
- exynos_drm_crtc_from_pipe(drm_dev, pipe)->pipe_clk = &hdata->phy_clk;
+ crtc = drm_crtc_from_index(drm_dev, pipe);
+ exynos_crtc = to_exynos_crtc(crtc);
+ exynos_crtc->pipe_clk = &hdata->phy_clk;
encoder->possible_crtcs = 1 << pipe;
.mode_set_nofb = fsl_dcu_drm_crtc_mode_set_nofb,
};
+static int fsl_dcu_drm_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ struct drm_device *dev = crtc->dev;
+ struct fsl_dcu_drm_device *fsl_dev = dev->dev_private;
+ unsigned int value;
+
+ regmap_read(fsl_dev->regmap, DCU_INT_MASK, &value);
+ value &= ~DCU_INT_MASK_VBLANK;
+ regmap_write(fsl_dev->regmap, DCU_INT_MASK, value);
+
+ return 0;
+}
+
+static void fsl_dcu_drm_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+ struct drm_device *dev = crtc->dev;
+ struct fsl_dcu_drm_device *fsl_dev = dev->dev_private;
+ unsigned int value;
+
+ regmap_read(fsl_dev->regmap, DCU_INT_MASK, &value);
+ value |= DCU_INT_MASK_VBLANK;
+ regmap_write(fsl_dev->regmap, DCU_INT_MASK, value);
+}
+
static const struct drm_crtc_funcs fsl_dcu_drm_crtc_funcs = {
.atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
.page_flip = drm_atomic_helper_page_flip,
.reset = drm_atomic_helper_crtc_reset,
.set_config = drm_atomic_helper_set_config,
+ .enable_vblank = fsl_dcu_drm_crtc_enable_vblank,
+ .disable_vblank = fsl_dcu_drm_crtc_disable_vblank,
};
int fsl_dcu_drm_crtc_create(struct fsl_dcu_drm_device *fsl_dev)
return IRQ_HANDLED;
}
-static int fsl_dcu_drm_enable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- struct fsl_dcu_drm_device *fsl_dev = dev->dev_private;
- unsigned int value;
-
- regmap_read(fsl_dev->regmap, DCU_INT_MASK, &value);
- value &= ~DCU_INT_MASK_VBLANK;
- regmap_write(fsl_dev->regmap, DCU_INT_MASK, value);
-
- return 0;
-}
-
-static void fsl_dcu_drm_disable_vblank(struct drm_device *dev,
- unsigned int pipe)
-{
- struct fsl_dcu_drm_device *fsl_dev = dev->dev_private;
- unsigned int value;
-
- regmap_read(fsl_dev->regmap, DCU_INT_MASK, &value);
- value |= DCU_INT_MASK_VBLANK;
- regmap_write(fsl_dev->regmap, DCU_INT_MASK, value);
-}
-
static void fsl_dcu_drm_lastclose(struct drm_device *dev)
{
struct fsl_dcu_drm_device *fsl_dev = dev->dev_private;
drm_fbdev_cma_restore_mode(fsl_dev->fbdev);
}
-static const struct file_operations fsl_dcu_drm_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .poll = drm_poll,
- .read = drm_read,
- .llseek = no_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(fsl_dcu_drm_fops);
static struct drm_driver fsl_dcu_drm_driver = {
.driver_features = DRIVER_HAVE_IRQ | DRIVER_GEM | DRIVER_MODESET
.load = fsl_dcu_load,
.unload = fsl_dcu_unload,
.irq_handler = fsl_dcu_drm_irq,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = fsl_dcu_drm_enable_vblank,
- .disable_vblank = fsl_dcu_drm_disable_vblank,
.gem_free_object_unlocked = drm_gem_cma_free_object,
.gem_vm_ops = &drm_gem_cma_vm_ops,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
head) {
if (tmp_encoder != encoder
&& tmp_encoder->crtc == encoder->crtc) {
- printk(KERN_ERR "Can't enable LVDS and another "
- "encoder on the same pipe\n");
+ pr_err("Can't enable LVDS and another encoder on the same pipe\n");
return false;
}
}
failed_find:
mutex_unlock(&dev->mode_config.mutex);
- printk(KERN_ERR "Failed find\n");
+ pr_err("Failed find\n");
psb_intel_i2c_destroy(gma_encoder->ddc_bus);
failed_ddc:
- printk(KERN_ERR "Failed DDC\n");
+ pr_err("Failed DDC\n");
psb_intel_i2c_destroy(gma_encoder->i2c_bus);
failed_blc_i2c:
- printk(KERN_ERR "Failed BLC\n");
+ pr_err("Failed BLC\n");
drm_encoder_cleanup(encoder);
drm_connector_cleanup(connector);
kfree(lvds_priv);
info = drm_fb_helper_alloc_fbi(&fbdev->psb_fb_helper);
if (IS_ERR(info)) {
ret = PTR_ERR(info);
- goto err_free_range;
+ goto out;
}
info->par = fbdev;
ret = psb_framebuffer_init(dev, psbfb, &mode_cmd, backing);
if (ret)
- goto err_release;
+ goto out;
fb = &psbfb->base;
psbfb->fbdev = info;
psbfb->base.width, psbfb->base.height);
return 0;
-err_release:
- drm_fb_helper_release_fbi(&fbdev->psb_fb_helper);
-err_free_range:
+out:
psb_gtt_free_range(dev, backing);
return ret;
}
struct psb_framebuffer *psbfb = &fbdev->pfb;
drm_fb_helper_unregister_fbi(&fbdev->psb_fb_helper);
- drm_fb_helper_release_fbi(&fbdev->psb_fb_helper);
drm_fb_helper_fini(&fbdev->psb_fb_helper);
drm_framebuffer_unregister_private(&psbfb->base);
((ti->vblank_hi << 8) | ti->vblank_lo);
mode->clock = ti->pixel_clock * 10;
#if 0
- printk(KERN_INFO "hdisplay is %d\n", mode->hdisplay);
- printk(KERN_INFO "vdisplay is %d\n", mode->vdisplay);
- printk(KERN_INFO "HSS is %d\n", mode->hsync_start);
- printk(KERN_INFO "HSE is %d\n", mode->hsync_end);
- printk(KERN_INFO "htotal is %d\n", mode->htotal);
- printk(KERN_INFO "VSS is %d\n", mode->vsync_start);
- printk(KERN_INFO "VSE is %d\n", mode->vsync_end);
- printk(KERN_INFO "vtotal is %d\n", mode->vtotal);
- printk(KERN_INFO "clock is %d\n", mode->clock);
+ pr_info("hdisplay is %d\n", mode->hdisplay);
+ pr_info("vdisplay is %d\n", mode->vdisplay);
+ pr_info("HSS is %d\n", mode->hsync_start);
+ pr_info("HSE is %d\n", mode->hsync_end);
+ pr_info("htotal is %d\n", mode->htotal);
+ pr_info("VSS is %d\n", mode->vsync_start);
+ pr_info("VSE is %d\n", mode->vsync_end);
+ pr_info("vtotal is %d\n", mode->vtotal);
+ pr_info("clock is %d\n", mode->clock);
#endif
mode_dev->panel_fixed_mode = mode;
}
#define PSB_RSGX32(_offs) \
({ \
if (inl(dev_priv->apm_base + PSB_APM_STS) & 0x3) { \
- printk(KERN_ERR \
- "access sgx when it's off!! (READ) %s, %d\n", \
- __FILE__, __LINE__); \
+ pr_err("access sgx when it's off!! (READ) %s, %d\n", \
+ __FILE__, __LINE__); \
melay(1000); \
} \
ioread32(dev_priv->sgx_reg + (_offs)); \
/* PSB requires the LVDS is on pipe B, MRST has only one pipe anyway */
if (!IS_MRST(dev) && gma_crtc->pipe == 0) {
- printk(KERN_ERR "Can't support LVDS on pipe A\n");
+ pr_err("Can't support LVDS on pipe A\n");
return false;
}
if (IS_MRST(dev) && gma_crtc->pipe != 0) {
- printk(KERN_ERR "Must use PIPE A\n");
+ pr_err("Must use PIPE A\n");
return false;
}
/* Should never happen!! */
head) {
if (tmp_encoder != encoder
&& tmp_encoder->crtc == encoder->crtc) {
- printk(KERN_ERR "Can't enable LVDS and another "
- "encoder on the same pipe\n");
+ pr_err("Can't enable LVDS and another encoder on the same pipe\n");
return false;
}
}
spin_unlock_irqrestore(&crtc->dev->event_lock, flags);
}
+static int hibmc_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ struct hibmc_drm_private *priv = crtc->dev->dev_private;
+
+ writel(HIBMC_RAW_INTERRUPT_EN_VBLANK(1),
+ priv->mmio + HIBMC_RAW_INTERRUPT_EN);
+
+ return 0;
+}
+
+static void hibmc_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+ struct hibmc_drm_private *priv = crtc->dev->dev_private;
+
+ writel(HIBMC_RAW_INTERRUPT_EN_VBLANK(0),
+ priv->mmio + HIBMC_RAW_INTERRUPT_EN);
+}
+
static const struct drm_crtc_funcs hibmc_crtc_funcs = {
.page_flip = drm_atomic_helper_page_flip,
.set_config = drm_atomic_helper_set_config,
.reset = drm_atomic_helper_crtc_reset,
.atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .enable_vblank = hibmc_crtc_enable_vblank,
+ .disable_vblank = hibmc_crtc_disable_vblank,
};
static const struct drm_crtc_helper_funcs hibmc_crtc_helper_funcs = {
.llseek = no_llseek,
};
-static int hibmc_enable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- struct hibmc_drm_private *priv =
- (struct hibmc_drm_private *)dev->dev_private;
-
- writel(HIBMC_RAW_INTERRUPT_EN_VBLANK(1),
- priv->mmio + HIBMC_RAW_INTERRUPT_EN);
-
- return 0;
-}
-
-static void hibmc_disable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- struct hibmc_drm_private *priv =
- (struct hibmc_drm_private *)dev->dev_private;
-
- writel(HIBMC_RAW_INTERRUPT_EN_VBLANK(0),
- priv->mmio + HIBMC_RAW_INTERRUPT_EN);
-}
-
irqreturn_t hibmc_drm_interrupt(int irq, void *arg)
{
struct drm_device *dev = (struct drm_device *)arg;
.desc = "hibmc drm driver",
.major = 1,
.minor = 0,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = hibmc_enable_vblank,
- .disable_vblank = hibmc_disable_vblank,
.gem_free_object_unlocked = hibmc_gem_free_object,
.dumb_create = hibmc_dumb_create,
.dumb_map_offset = hibmc_dumb_mmap_offset,
return 0;
out_release_fbi:
- drm_fb_helper_release_fbi(helper);
ret1 = ttm_bo_reserve(&bo->bo, true, false, NULL);
if (ret1) {
DRM_ERROR("failed to rsv ttm_bo when release fbi: %d\n", ret1);
struct drm_fb_helper *fbh = &fbdev->helper;
drm_fb_helper_unregister_fbi(fbh);
- drm_fb_helper_release_fbi(fbh);
drm_fb_helper_fini(fbh);
SOCKET_QOS_EN, SOCKET_QOS_EN);
}
-static int ade_enable_vblank(struct drm_device *dev, unsigned int pipe)
+static int ade_crtc_enable_vblank(struct drm_crtc *crtc)
{
- struct drm_crtc *crtc = drm_crtc_from_index(dev, pipe);
struct ade_crtc *acrtc = to_ade_crtc(crtc);
struct ade_hw_ctx *ctx = acrtc->ctx;
void __iomem *base = ctx->base;
return 0;
}
-static void ade_disable_vblank(struct drm_device *dev, unsigned int pipe)
+static void ade_crtc_disable_vblank(struct drm_crtc *crtc)
{
- struct drm_crtc *crtc = drm_crtc_from_index(dev, pipe);
struct ade_crtc *acrtc = to_ade_crtc(crtc);
struct ade_hw_ctx *ctx = acrtc->ctx;
void __iomem *base = ctx->base;
.set_property = drm_atomic_helper_crtc_set_property,
.atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .enable_vblank = ade_crtc_enable_vblank,
+ .disable_vblank = ade_crtc_disable_vblank,
};
static int ade_crtc_init(struct drm_device *dev, struct drm_crtc *crtc,
IRQF_SHARED, dev->driver->name, acrtc);
if (ret)
return ret;
- dev->driver->get_vblank_counter = drm_vblank_no_hw_counter;
- dev->driver->enable_vblank = ade_enable_vblank;
- dev->driver->disable_vblank = ade_disable_vblank;
return 0;
}
return ret;
}
-static const struct file_operations kirin_drm_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .poll = drm_poll,
- .read = drm_read,
- .llseek = no_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(kirin_drm_fops);
static int kirin_gem_cma_dumb_create(struct drm_file *file,
struct drm_device *dev,
select INPUT if ACPI
select ACPI_VIDEO if ACPI
select ACPI_BUTTON if ACPI
+ select SYNC_FILE
+ select IOSF_MBI
help
Choose this option if you have a system that has "Intel Graphics
Media Accelerator" or "HD Graphics" integrated graphics,
select X86_MSR # used by igt/pm_rpm
select DRM_VGEM # used by igt/prime_vgem (dmabuf interop checks)
select DRM_DEBUG_MM if DRM=y
+ select DRM_DEBUG_MM_SELFTEST
select DRM_I915_SW_FENCE_DEBUG_OBJECTS
+ select DRM_I915_SELFTEST
default n
help
Choose this option to turn on extra driver debugging that may affect
Recommended for driver developers only.
If in doubt, say "N".
+
+config DRM_I915_SELFTEST
+ bool "Enable selftests upon driver load"
+ depends on DRM_I915
+ default n
+ select FAULT_INJECTION
+ select PRIME_NUMBERS
+ help
+ Choose this option to allow the driver to perform selftests upon
+ loading; also requires the i915.selftest=1 module parameter. To
+ exit the module after running the selftests (i.e. to prevent normal
+ module initialisation afterwards) use i915.selftest=-1.
+
+ Recommended for driver developers only.
+
+ If in doubt, say "N".
+
+config DRM_I915_LOW_LEVEL_TRACEPOINTS
+ bool "Enable low level request tracing events"
+ depends on DRM_I915
+ default n
+ help
+ Choose this option to turn on low level request tracing events.
+ This provides the ability to precisely monitor engine utilisation
+ and also analyze the request dependency resolving timeline.
+
+ If in doubt, say "N".
# GEM code
i915-y += i915_cmd_parser.o \
i915_gem_batch_pool.o \
+ i915_gem_clflush.o \
i915_gem_context.o \
i915_gem_dmabuf.o \
i915_gem_evict.o \
intel_atomic.o \
intel_atomic_plane.o \
intel_bios.o \
+ intel_cdclk.o \
intel_color.o \
intel_display.o \
intel_dpio_phy.o \
intel_dp.o \
intel_dsi.o \
intel_dsi_dcs_backlight.o \
- intel_dsi_panel_vbt.o \
intel_dsi_pll.o \
+ intel_dsi_vbt.o \
intel_dvo.o \
intel_hdmi.o \
intel_i2c.o \
# Post-mortem debug and GPU hang state capture
i915-$(CONFIG_DRM_I915_CAPTURE_ERROR) += i915_gpu_error.o
+i915-$(CONFIG_DRM_I915_SELFTEST) += \
+ selftests/i915_random.o \
+ selftests/i915_selftest.o
# virtual gpu code
i915-y += i915_vgpu.o
len += copy_len;
gma += copy_len;
}
- return 0;
+ return len;
}
static int shadow_workload_ring_buffer(struct intel_vgpu_workload *workload)
{
struct intel_vgpu *vgpu = workload->vgpu;
- int ring_id = workload->ring_id;
- struct i915_gem_context *shadow_ctx = vgpu->shadow_ctx;
- struct intel_ring *ring = shadow_ctx->engine[ring_id].ring;
unsigned long gma_head, gma_tail, gma_top, guest_rb_size;
- unsigned int copy_len = 0;
+ u32 *cs;
int ret;
guest_rb_size = _RING_CTL_BUF_SIZE(workload->rb_ctl);
gma_top = workload->rb_start + guest_rb_size;
/* allocate shadow ring buffer */
- ret = intel_ring_begin(workload->req, workload->rb_len / 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(workload->req, workload->rb_len / sizeof(u32));
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
/* get shadow ring buffer va */
- workload->shadow_ring_buffer_va = ring->vaddr + ring->tail;
+ workload->shadow_ring_buffer_va = cs;
/* head > tail --> copy head <-> top */
if (gma_head > gma_tail) {
ret = copy_gma_to_hva(vgpu, vgpu->gtt.ggtt_mm,
- gma_head, gma_top,
- workload->shadow_ring_buffer_va);
+ gma_head, gma_top, cs);
if (ret) {
gvt_vgpu_err("fail to copy guest ring buffer\n");
return ret;
}
- copy_len = gma_top - gma_head;
+ cs += ret / sizeof(u32);
gma_head = workload->rb_start;
}
/* copy head or start <-> tail */
- ret = copy_gma_to_hva(vgpu, vgpu->gtt.ggtt_mm,
- gma_head, gma_tail,
- workload->shadow_ring_buffer_va + copy_len);
+ ret = copy_gma_to_hva(vgpu, vgpu->gtt.ggtt_mm, gma_head, gma_tail, cs);
if (ret) {
gvt_vgpu_err("fail to copy guest ring buffer\n");
return ret;
}
- ring->tail += workload->rb_len;
- intel_ring_advance(ring);
+ cs += ret / sizeof(u32);
+ intel_ring_advance(workload->req, cs);
return 0;
}
workload->status = ret;
if (!IS_ERR_OR_NULL(rq))
- i915_add_request_no_flush(rq);
+ i915_add_request(rq);
else
engine->context_unpin(engine, shadow_ctx);
* space. Parsing should be faster in some cases this way.
*/
batch_end = cmd + (batch_len / sizeof(*batch_end));
- while (cmd < batch_end) {
+ do {
u32 length;
- if (*cmd == MI_BATCH_BUFFER_END)
+ if (*cmd == MI_BATCH_BUFFER_END) {
+ if (needs_clflush_after) {
+ void *ptr = ptr_mask_bits(shadow_batch_obj->mm.mapping);
+ drm_clflush_virt_range(ptr,
+ (void *)(cmd + 1) - ptr);
+ }
break;
+ }
desc = find_cmd(engine, *cmd, desc, &default_desc);
if (!desc) {
}
cmd += length;
- }
-
- if (cmd >= batch_end) {
- DRM_DEBUG_DRIVER("CMD: Got to the end of the buffer w/o a BBE cmd!\n");
- ret = -EINVAL;
- }
+ if (cmd >= batch_end) {
+ DRM_DEBUG_DRIVER("CMD: Got to the end of the buffer w/o a BBE cmd!\n");
+ ret = -EINVAL;
+ break;
+ }
+ } while (1);
- if (ret == 0 && needs_clflush_after)
- drm_clflush_virt_range(shadow_batch_obj->mm.mapping, batch_len);
i915_gem_object_unpin_map(shadow_batch_obj);
-
return ret;
}
*/
#include <linux/debugfs.h>
-#include <linux/list_sort.h>
+#include <linux/sort.h>
#include "intel_drv.h"
static inline struct drm_i915_private *node_to_i915(struct drm_info_node *node)
return to_i915(node->minor->dev);
}
-/* As the drm_debugfs_init() routines are called before dev->dev_private is
- * allocated we need to hook into the minor for release. */
-static int
-drm_add_fake_info_node(struct drm_minor *minor,
- struct dentry *ent,
- const void *key)
+static __always_inline void seq_print_param(struct seq_file *m,
+ const char *name,
+ const char *type,
+ const void *x)
{
- struct drm_info_node *node;
-
- node = kmalloc(sizeof(*node), GFP_KERNEL);
- if (node == NULL) {
- debugfs_remove(ent);
- return -ENOMEM;
- }
-
- node->minor = minor;
- node->dent = ent;
- node->info_ent = (void *)key;
-
- mutex_lock(&minor->debugfs_lock);
- list_add(&node->list, &minor->debugfs_list);
- mutex_unlock(&minor->debugfs_lock);
-
- return 0;
+ if (!__builtin_strcmp(type, "bool"))
+ seq_printf(m, "i915.%s=%s\n", name, yesno(*(const bool *)x));
+ else if (!__builtin_strcmp(type, "int"))
+ seq_printf(m, "i915.%s=%d\n", name, *(const int *)x);
+ else if (!__builtin_strcmp(type, "unsigned int"))
+ seq_printf(m, "i915.%s=%u\n", name, *(const unsigned int *)x);
+ else if (!__builtin_strcmp(type, "char *"))
+ seq_printf(m, "i915.%s=%s\n", name, *(const char **)x);
+ else
+ BUILD_BUG();
}
static int i915_capabilities(struct seq_file *m, void *data)
seq_printf(m, "gen: %d\n", INTEL_GEN(dev_priv));
seq_printf(m, "platform: %s\n", intel_platform_name(info->platform));
seq_printf(m, "pch: %d\n", INTEL_PCH_TYPE(dev_priv));
+
#define PRINT_FLAG(x) seq_printf(m, #x ": %s\n", yesno(info->x))
DEV_INFO_FOR_EACH_FLAG(PRINT_FLAG);
#undef PRINT_FLAG
+ kernel_param_lock(THIS_MODULE);
+#define PRINT_PARAM(T, x) seq_print_param(m, #x, #T, &i915.x);
+ I915_PARAMS_FOR_EACH(PRINT_PARAM);
+#undef PRINT_PARAM
+ kernel_param_unlock(THIS_MODULE);
+
return 0;
}
seq_printf(m, " (frontbuffer: 0x%03x)", frontbuffer_bits);
}
-static int obj_rank_by_stolen(void *priv,
- struct list_head *A, struct list_head *B)
+static int obj_rank_by_stolen(const void *A, const void *B)
{
- struct drm_i915_gem_object *a =
- container_of(A, struct drm_i915_gem_object, obj_exec_link);
- struct drm_i915_gem_object *b =
- container_of(B, struct drm_i915_gem_object, obj_exec_link);
+ const struct drm_i915_gem_object *a =
+ *(const struct drm_i915_gem_object **)A;
+ const struct drm_i915_gem_object *b =
+ *(const struct drm_i915_gem_object **)B;
if (a->stolen->start < b->stolen->start)
return -1;
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
struct drm_device *dev = &dev_priv->drm;
+ struct drm_i915_gem_object **objects;
struct drm_i915_gem_object *obj;
u64 total_obj_size, total_gtt_size;
- LIST_HEAD(stolen);
- int count, ret;
+ unsigned long total, count, n;
+ int ret;
+
+ total = READ_ONCE(dev_priv->mm.object_count);
+ objects = drm_malloc_ab(total, sizeof(*objects));
+ if (!objects)
+ return -ENOMEM;
ret = mutex_lock_interruptible(&dev->struct_mutex);
if (ret)
- return ret;
+ goto out;
total_obj_size = total_gtt_size = count = 0;
list_for_each_entry(obj, &dev_priv->mm.bound_list, global_link) {
+ if (count == total)
+ break;
+
if (obj->stolen == NULL)
continue;
- list_add(&obj->obj_exec_link, &stolen);
-
+ objects[count++] = obj;
total_obj_size += obj->base.size;
total_gtt_size += i915_gem_obj_total_ggtt_size(obj);
- count++;
+
}
list_for_each_entry(obj, &dev_priv->mm.unbound_list, global_link) {
+ if (count == total)
+ break;
+
if (obj->stolen == NULL)
continue;
- list_add(&obj->obj_exec_link, &stolen);
-
+ objects[count++] = obj;
total_obj_size += obj->base.size;
- count++;
}
- list_sort(NULL, &stolen, obj_rank_by_stolen);
+
+ sort(objects, count, sizeof(*objects), obj_rank_by_stolen, NULL);
+
seq_puts(m, "Stolen:\n");
- while (!list_empty(&stolen)) {
- obj = list_first_entry(&stolen, typeof(*obj), obj_exec_link);
+ for (n = 0; n < count; n++) {
seq_puts(m, " ");
- describe_obj(m, obj);
+ describe_obj(m, objects[n]);
seq_putc(m, '\n');
- list_del_init(&obj->obj_exec_link);
}
- mutex_unlock(&dev->struct_mutex);
-
- seq_printf(m, "Total %d objects, %llu bytes, %llu GTT size\n",
+ seq_printf(m, "Total %lu objects, %llu bytes, %llu GTT size\n",
count, total_obj_size, total_gtt_size);
- return 0;
+
+ mutex_unlock(&dev->struct_mutex);
+out:
+ drm_free_large(objects);
+ return ret;
}
struct file_stats {
dpy_count, dpy_size);
seq_printf(m, "%llu [%llu] gtt total\n",
- ggtt->base.total, ggtt->mappable_end - ggtt->base.start);
+ ggtt->base.total, ggtt->mappable_end);
seq_putc(m, '\n');
print_batch_pool_stats(m, dev_priv);
mutex_lock(&dev->struct_mutex);
request = list_first_entry_or_null(&file_priv->mm.request_list,
struct drm_i915_gem_request,
- client_list);
+ client_link);
rcu_read_lock();
task = pid_task(request && request->ctx->pid ?
request->ctx->pid : file->pid,
seq_printf(m, "Current sequence (%s): %x\n",
engine->name, intel_engine_get_seqno(engine));
- spin_lock_irq(&b->lock);
+ spin_lock_irq(&b->rb_lock);
for (rb = rb_first(&b->waiters); rb; rb = rb_next(rb)) {
struct intel_wait *w = rb_entry(rb, typeof(*w), node);
seq_printf(m, "Waiting (%s): %s [%d] on %x\n",
engine->name, w->tsk->comm, w->tsk->pid, w->seqno);
}
- spin_unlock_irq(&b->lock);
+ spin_unlock_irq(&b->rb_lock);
}
static int i915_gem_seqno_info(struct seq_file *m, void *data)
I915_READ(VLV_IIR_RW));
seq_printf(m, "Display IMR:\t%08x\n",
I915_READ(VLV_IMR));
- for_each_pipe(dev_priv, pipe)
+ for_each_pipe(dev_priv, pipe) {
+ enum intel_display_power_domain power_domain;
+
+ power_domain = POWER_DOMAIN_PIPE(pipe);
+ if (!intel_display_power_get_if_enabled(dev_priv,
+ power_domain)) {
+ seq_printf(m, "Pipe %c power disabled\n",
+ pipe_name(pipe));
+ continue;
+ }
+
seq_printf(m, "Pipe %c stat:\t%08x\n",
pipe_name(pipe),
I915_READ(PIPESTAT(pipe)));
+ intel_display_power_put(dev_priv, power_domain);
+ }
seq_printf(m, "Master IER:\t%08x\n",
I915_READ(VLV_MASTER_IER));
}
#if IS_ENABLED(CONFIG_DRM_I915_CAPTURE_ERROR)
-
-static ssize_t
-i915_error_state_write(struct file *filp,
- const char __user *ubuf,
- size_t cnt,
- loff_t *ppos)
+static ssize_t gpu_state_read(struct file *file, char __user *ubuf,
+ size_t count, loff_t *pos)
{
- struct i915_error_state_file_priv *error_priv = filp->private_data;
-
- DRM_DEBUG_DRIVER("Resetting error state\n");
- i915_destroy_error_state(error_priv->i915);
-
- return cnt;
-}
+ struct i915_gpu_state *error = file->private_data;
+ struct drm_i915_error_state_buf str;
+ ssize_t ret;
+ loff_t tmp;
-static int i915_error_state_open(struct inode *inode, struct file *file)
-{
- struct drm_i915_private *dev_priv = inode->i_private;
- struct i915_error_state_file_priv *error_priv;
+ if (!error)
+ return 0;
- error_priv = kzalloc(sizeof(*error_priv), GFP_KERNEL);
- if (!error_priv)
- return -ENOMEM;
+ ret = i915_error_state_buf_init(&str, error->i915, count, *pos);
+ if (ret)
+ return ret;
- error_priv->i915 = dev_priv;
+ ret = i915_error_state_to_str(&str, error);
+ if (ret)
+ goto out;
- i915_error_state_get(&dev_priv->drm, error_priv);
+ tmp = 0;
+ ret = simple_read_from_buffer(ubuf, count, &tmp, str.buf, str.bytes);
+ if (ret < 0)
+ goto out;
- file->private_data = error_priv;
+ *pos = str.start + ret;
+out:
+ i915_error_state_buf_release(&str);
+ return ret;
+}
+static int gpu_state_release(struct inode *inode, struct file *file)
+{
+ i915_gpu_state_put(file->private_data);
return 0;
}
-static int i915_error_state_release(struct inode *inode, struct file *file)
+static int i915_gpu_info_open(struct inode *inode, struct file *file)
{
- struct i915_error_state_file_priv *error_priv = file->private_data;
+ struct i915_gpu_state *gpu;
- i915_error_state_put(error_priv);
- kfree(error_priv);
+ gpu = i915_capture_gpu_state(inode->i_private);
+ if (!gpu)
+ return -ENOMEM;
+ file->private_data = gpu;
return 0;
}
-static ssize_t i915_error_state_read(struct file *file, char __user *userbuf,
- size_t count, loff_t *pos)
+static const struct file_operations i915_gpu_info_fops = {
+ .owner = THIS_MODULE,
+ .open = i915_gpu_info_open,
+ .read = gpu_state_read,
+ .llseek = default_llseek,
+ .release = gpu_state_release,
+};
+
+static ssize_t
+i915_error_state_write(struct file *filp,
+ const char __user *ubuf,
+ size_t cnt,
+ loff_t *ppos)
{
- struct i915_error_state_file_priv *error_priv = file->private_data;
- struct drm_i915_error_state_buf error_str;
- loff_t tmp_pos = 0;
- ssize_t ret_count = 0;
- int ret;
+ struct i915_gpu_state *error = filp->private_data;
- ret = i915_error_state_buf_init(&error_str, error_priv->i915,
- count, *pos);
- if (ret)
- return ret;
+ if (!error)
+ return 0;
- ret = i915_error_state_to_str(&error_str, error_priv);
- if (ret)
- goto out;
+ DRM_DEBUG_DRIVER("Resetting error state\n");
+ i915_reset_error_state(error->i915);
- ret_count = simple_read_from_buffer(userbuf, count, &tmp_pos,
- error_str.buf,
- error_str.bytes);
+ return cnt;
+}
- if (ret_count < 0)
- ret = ret_count;
- else
- *pos = error_str.start + ret_count;
-out:
- i915_error_state_buf_release(&error_str);
- return ret ?: ret_count;
+static int i915_error_state_open(struct inode *inode, struct file *file)
+{
+ file->private_data = i915_first_error_state(inode->i_private);
+ return 0;
}
static const struct file_operations i915_error_state_fops = {
.owner = THIS_MODULE,
.open = i915_error_state_open,
- .read = i915_error_state_read,
+ .read = gpu_state_read,
.write = i915_error_state_write,
.llseek = default_llseek,
- .release = i915_error_state_release,
+ .release = gpu_state_release,
};
-
#endif
-static int
-i915_next_seqno_get(void *data, u64 *val)
-{
- struct drm_i915_private *dev_priv = data;
-
- *val = 1 + atomic_read(&dev_priv->gt.global_timeline.seqno);
- return 0;
-}
-
static int
i915_next_seqno_set(void *data, u64 val)
{
}
DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops,
- i915_next_seqno_get, i915_next_seqno_set,
+ NULL, i915_next_seqno_set,
"0x%llx\n");
static int i915_frequency_info(struct seq_file *m, void *unused)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct drm_device *dev = &dev_priv->drm;
int ret = 0;
intel_runtime_pm_get(dev_priv);
}
/* RPSTAT1 is in the GT power well */
- ret = mutex_lock_interruptible(&dev->struct_mutex);
- if (ret)
- goto out;
-
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
reqf = I915_READ(GEN6_RPNSWREQ);
cagf = intel_gpu_freq(dev_priv, cagf);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
- mutex_unlock(&dev->struct_mutex);
if (IS_GEN6(dev_priv) || IS_GEN7(dev_priv)) {
pm_ier = I915_READ(GEN6_PMIER);
}
seq_printf(m, "PM IER=0x%08x IMR=0x%08x ISR=0x%08x IIR=0x%08x, MASK=0x%08x\n",
pm_ier, pm_imr, pm_isr, pm_iir, pm_mask);
- seq_printf(m, "pm_intr_keep: 0x%08x\n", dev_priv->rps.pm_intr_keep);
+ seq_printf(m, "pm_intrmsk_mbz: 0x%08x\n",
+ dev_priv->rps.pm_intrmsk_mbz);
seq_printf(m, "GT_PERF_STATUS: 0x%08x\n", gt_perf_status);
seq_printf(m, "Render p-state ratio: %d\n",
(gt_perf_status & (IS_GEN9(dev_priv) ? 0x1ff00 : 0xff00)) >> 8);
max_freq = (IS_GEN9_LP(dev_priv) ? rp_state_cap >> 0 :
rp_state_cap >> 16) & 0xff;
- max_freq *= (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv) ?
- GEN9_FREQ_SCALER : 1);
+ max_freq *= (IS_GEN9_BC(dev_priv) ? GEN9_FREQ_SCALER : 1);
seq_printf(m, "Lowest (RPN) frequency: %dMHz\n",
intel_gpu_freq(dev_priv, max_freq));
max_freq = (rp_state_cap & 0xff00) >> 8;
- max_freq *= (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv) ?
- GEN9_FREQ_SCALER : 1);
+ max_freq *= (IS_GEN9_BC(dev_priv) ? GEN9_FREQ_SCALER : 1);
seq_printf(m, "Nominal (RP1) frequency: %dMHz\n",
intel_gpu_freq(dev_priv, max_freq));
max_freq = (IS_GEN9_LP(dev_priv) ? rp_state_cap >> 16 :
rp_state_cap >> 0) & 0xff;
- max_freq *= (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv) ?
- GEN9_FREQ_SCALER : 1);
+ max_freq *= (IS_GEN9_BC(dev_priv) ? GEN9_FREQ_SCALER : 1);
seq_printf(m, "Max non-overclocked (RP0) frequency: %dMHz\n",
intel_gpu_freq(dev_priv, max_freq));
seq_printf(m, "Max overclocked frequency: %dMHz\n",
seq_puts(m, "no P-state info available\n");
}
- seq_printf(m, "Current CD clock frequency: %d kHz\n", dev_priv->cdclk_freq);
+ seq_printf(m, "Current CD clock frequency: %d kHz\n", dev_priv->cdclk.hw.cdclk);
seq_printf(m, "Max CD clock frequency: %d kHz\n", dev_priv->max_cdclk_freq);
seq_printf(m, "Max pixel clock frequency: %d kHz\n", dev_priv->max_dotclk_freq);
-out:
intel_runtime_pm_put(dev_priv);
return ret;
}
enum intel_engine_id id;
if (test_bit(I915_WEDGED, &dev_priv->gpu_error.flags))
- seq_printf(m, "Wedged\n");
- if (test_bit(I915_RESET_IN_PROGRESS, &dev_priv->gpu_error.flags))
- seq_printf(m, "Reset in progress\n");
+ seq_puts(m, "Wedged\n");
+ if (test_bit(I915_RESET_BACKOFF, &dev_priv->gpu_error.flags))
+ seq_puts(m, "Reset in progress: struct_mutex backoff\n");
+ if (test_bit(I915_RESET_HANDOFF, &dev_priv->gpu_error.flags))
+ seq_puts(m, "Reset in progress: reset handoff to waiter\n");
if (waitqueue_active(&dev_priv->gpu_error.wait_queue))
- seq_printf(m, "Waiter holding struct mutex\n");
+ seq_puts(m, "Waiter holding struct mutex\n");
if (waitqueue_active(&dev_priv->gpu_error.reset_queue))
- seq_printf(m, "struct_mutex blocked for reset\n");
+ seq_puts(m, "struct_mutex blocked for reset\n");
if (!i915.enable_hangcheck) {
- seq_printf(m, "Hangcheck disabled\n");
+ seq_puts(m, "Hangcheck disabled\n");
return 0;
}
intel_runtime_pm_put(dev_priv);
- if (delayed_work_pending(&dev_priv->gpu_error.hangcheck_work)) {
- seq_printf(m, "Hangcheck active, fires in %dms\n",
+ if (timer_pending(&dev_priv->gpu_error.hangcheck_work.timer))
+ seq_printf(m, "Hangcheck active, timer fires in %dms\n",
jiffies_to_msecs(dev_priv->gpu_error.hangcheck_work.timer.expires -
jiffies));
- } else
- seq_printf(m, "Hangcheck inactive\n");
+ else if (delayed_work_pending(&dev_priv->gpu_error.hangcheck_work))
+ seq_puts(m, "Hangcheck active, work pending\n");
+ else
+ seq_puts(m, "Hangcheck inactive\n");
+
+ seq_printf(m, "GT active? %s\n", yesno(dev_priv->gt.awake));
for_each_engine(engine, dev_priv, id) {
struct intel_breadcrumbs *b = &engine->breadcrumbs;
struct rb_node *rb;
seq_printf(m, "%s:\n", engine->name);
- seq_printf(m, "\tseqno = %x [current %x, last %x]\n",
+ seq_printf(m, "\tseqno = %x [current %x, last %x], inflight %d\n",
engine->hangcheck.seqno, seqno[id],
- intel_engine_last_submit(engine));
+ intel_engine_last_submit(engine),
+ engine->timeline->inflight_seqnos);
seq_printf(m, "\twaiters? %s, fake irq active? %s, stalled? %s\n",
yesno(intel_engine_has_waiter(engine)),
yesno(test_bit(engine->id,
&dev_priv->gpu_error.missed_irq_rings)),
yesno(engine->hangcheck.stalled));
- spin_lock_irq(&b->lock);
+ spin_lock_irq(&b->rb_lock);
for (rb = rb_first(&b->waiters); rb; rb = rb_next(rb)) {
struct intel_wait *w = rb_entry(rb, typeof(*w), node);
seq_printf(m, "\t%s [%d] waiting for %x\n",
w->tsk->comm, w->tsk->pid, w->seqno);
}
- spin_unlock_irq(&b->lock);
+ spin_unlock_irq(&b->rb_lock);
seq_printf(m, "\tACTHD = 0x%08llx [current 0x%08llx]\n",
(long long)engine->hangcheck.acthd,
u32 rgvmodectl, rstdbyctl;
u16 crstandvid;
- intel_runtime_pm_get(dev_priv);
-
rgvmodectl = I915_READ(MEMMODECTL);
rstdbyctl = I915_READ(RSTDBYCTL);
crstandvid = I915_READ16(CRSTANDVID);
- intel_runtime_pm_put(dev_priv);
-
seq_printf(m, "HD boost: %s\n", yesno(rgvmodectl & MEMMODE_BOOST_EN));
seq_printf(m, "Boost freq: %d\n",
(rgvmodectl & MEMMODE_BOOST_FREQ_MASK) >>
return 0;
}
+static void print_rc6_res(struct seq_file *m,
+ const char *title,
+ const i915_reg_t reg)
+{
+ struct drm_i915_private *dev_priv = node_to_i915(m->private);
+
+ seq_printf(m, "%s %u (%llu us)\n",
+ title, I915_READ(reg),
+ intel_rc6_residency_us(dev_priv, reg));
+}
+
static int vlv_drpc_info(struct seq_file *m)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
u32 rpmodectl1, rcctl1, pw_status;
- intel_runtime_pm_get(dev_priv);
-
pw_status = I915_READ(VLV_GTLC_PW_STATUS);
rpmodectl1 = I915_READ(GEN6_RP_CONTROL);
rcctl1 = I915_READ(GEN6_RC_CONTROL);
- intel_runtime_pm_put(dev_priv);
-
seq_printf(m, "Video Turbo Mode: %s\n",
yesno(rpmodectl1 & GEN6_RP_MEDIA_TURBO));
seq_printf(m, "Turbo enabled: %s\n",
seq_printf(m, "Media Power Well: %s\n",
(pw_status & VLV_GTLC_PW_MEDIA_STATUS_MASK) ? "Up" : "Down");
- seq_printf(m, "Render RC6 residency since boot: %u\n",
- I915_READ(VLV_GT_RENDER_RC6));
- seq_printf(m, "Media RC6 residency since boot: %u\n",
- I915_READ(VLV_GT_MEDIA_RC6));
+ print_rc6_res(m, "Render RC6 residency since boot:", VLV_GT_RENDER_RC6);
+ print_rc6_res(m, "Media RC6 residency since boot:", VLV_GT_MEDIA_RC6);
return i915_forcewake_domains(m, NULL);
}
static int gen6_drpc_info(struct seq_file *m)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct drm_device *dev = &dev_priv->drm;
u32 rpmodectl1, gt_core_status, rcctl1, rc6vids = 0;
u32 gen9_powergate_enable = 0, gen9_powergate_status = 0;
unsigned forcewake_count;
- int count = 0, ret;
-
- ret = mutex_lock_interruptible(&dev->struct_mutex);
- if (ret)
- return ret;
- intel_runtime_pm_get(dev_priv);
-
- spin_lock_irq(&dev_priv->uncore.lock);
- forcewake_count = dev_priv->uncore.fw_domain[FW_DOMAIN_ID_RENDER].wake_count;
- spin_unlock_irq(&dev_priv->uncore.lock);
+ int count = 0;
+ forcewake_count = READ_ONCE(dev_priv->uncore.fw_domain[FW_DOMAIN_ID_RENDER].wake_count);
if (forcewake_count) {
seq_puts(m, "RC information inaccurate because somebody "
"holds a forcewake reference \n");
gen9_powergate_enable = I915_READ(GEN9_PG_ENABLE);
gen9_powergate_status = I915_READ(GEN9_PWRGT_DOMAIN_STATUS);
}
- mutex_unlock(&dev->struct_mutex);
+
mutex_lock(&dev_priv->rps.hw_lock);
sandybridge_pcode_read(dev_priv, GEN6_PCODE_READ_RC6VIDS, &rc6vids);
mutex_unlock(&dev_priv->rps.hw_lock);
- intel_runtime_pm_put(dev_priv);
-
seq_printf(m, "Video Turbo Mode: %s\n",
yesno(rpmodectl1 & GEN6_RP_MEDIA_TURBO));
seq_printf(m, "HW control enabled: %s\n",
}
/* Not exactly sure what this is */
- seq_printf(m, "RC6 \"Locked to RPn\" residency since boot: %u\n",
- I915_READ(GEN6_GT_GFX_RC6_LOCKED));
- seq_printf(m, "RC6 residency since boot: %u\n",
- I915_READ(GEN6_GT_GFX_RC6));
- seq_printf(m, "RC6+ residency since boot: %u\n",
- I915_READ(GEN6_GT_GFX_RC6p));
- seq_printf(m, "RC6++ residency since boot: %u\n",
- I915_READ(GEN6_GT_GFX_RC6pp));
+ print_rc6_res(m, "RC6 \"Locked to RPn\" residency since boot:",
+ GEN6_GT_GFX_RC6_LOCKED);
+ print_rc6_res(m, "RC6 residency since boot:", GEN6_GT_GFX_RC6);
+ print_rc6_res(m, "RC6+ residency since boot:", GEN6_GT_GFX_RC6p);
+ print_rc6_res(m, "RC6++ residency since boot:", GEN6_GT_GFX_RC6pp);
seq_printf(m, "RC6 voltage: %dmV\n",
GEN6_DECODE_RC6_VID(((rc6vids >> 0) & 0xff)));
static int i915_drpc_info(struct seq_file *m, void *unused)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
+ int err;
+
+ intel_runtime_pm_get(dev_priv);
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
- return vlv_drpc_info(m);
+ err = vlv_drpc_info(m);
else if (INTEL_GEN(dev_priv) >= 6)
- return gen6_drpc_info(m);
+ err = gen6_drpc_info(m);
else
- return ironlake_drpc_info(m);
+ err = ironlake_drpc_info(m);
+
+ intel_runtime_pm_put(dev_priv);
+
+ return err;
}
static int i915_frontbuffer_tracking(struct seq_file *m, void *unused)
intel_runtime_pm_get(dev_priv);
intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
- if (HAS_PCH_SPLIT(dev_priv))
+ if (INTEL_GEN(dev_priv) >= 9)
+ /* no global SR status; inspect per-plane WM */;
+ else if (HAS_PCH_SPLIT(dev_priv))
sr_enabled = I915_READ(WM1_LP_ILK) & WM1_LP_SR_EN;
else if (IS_I965GM(dev_priv) || IS_G4X(dev_priv) ||
IS_I945G(dev_priv) || IS_I945GM(dev_priv))
if (ret)
goto out;
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ if (IS_GEN9_BC(dev_priv)) {
/* Convert GT frequency to 50 HZ units */
min_gpu_freq =
dev_priv->rps.min_freq_softlimit / GEN9_FREQ_SCALER;
&ia_freq);
seq_printf(m, "%d\t\t%d\t\t\t\t%d\n",
intel_gpu_freq(dev_priv, (gpu_freq *
- (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv) ?
- GEN9_FREQ_SCALER : 1))),
+ (IS_GEN9_BC(dev_priv) ?
+ GEN9_FREQ_SCALER : 1))),
((ia_freq >> 0) & 0xff) * 100,
((ia_freq >> 8) & 0xff) * 100);
}
seq_printf(m, "\nRPS Autotuning (current \"%s\" window):\n",
rps_power_to_str(dev_priv->rps.power));
seq_printf(m, " Avg. up: %d%% [above threshold? %d%%]\n",
- 100 * rpup / rpupei,
+ rpup && rpupei ? 100 * rpup / rpupei : 0,
dev_priv->rps.up_threshold);
seq_printf(m, " Avg. down: %d%% [below threshold? %d%%]\n",
- 100 * rpdown / rpdownei,
+ rpdown && rpdownei ? 100 * rpdown / rpdownei : 0,
dev_priv->rps.down_threshold);
} else {
seq_puts(m, "\nRPS Autotuning inactive\n");
seq_printf(m, "\tRSA: offset is %d; size = %d\n",
huc_fw->rsa_offset, huc_fw->rsa_size);
+ intel_runtime_pm_get(dev_priv);
seq_printf(m, "\nHuC status 0x%08x:\n", I915_READ(HUC_STATUS2));
+ intel_runtime_pm_put(dev_priv);
return 0;
}
seq_printf(m, "\tRSA: offset is %d; size = %d\n",
guc_fw->rsa_offset, guc_fw->rsa_size);
+ intel_runtime_pm_get(dev_priv);
+
tmp = I915_READ(GUC_STATUS);
seq_printf(m, "\nGuC status 0x%08x:\n", tmp);
for (i = 0; i < 16; i++)
seq_printf(m, "\t%2d: \t0x%x\n", i, I915_READ(SOFT_SCRATCH(i)));
+ intel_runtime_pm_put(dev_priv);
+
return 0;
}
struct drm_i915_private *dev_priv = node_to_i915(m->private);
struct drm_device *dev = &dev_priv->drm;
struct intel_connector *connector;
+ struct drm_connector_list_iter conn_iter;
struct intel_dp *intel_dp = NULL;
int ret;
u8 crc[6];
drm_modeset_lock_all(dev);
- for_each_intel_connector(dev, connector) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ for_each_intel_connector_iter(connector, &conn_iter) {
struct drm_crtc *crtc;
if (!connector->base.state->best_encoder)
}
ret = -ENODEV;
out:
+ drm_connector_list_iter_end(&conn_iter);
drm_modeset_unlock_all(dev);
return ret;
}
seq_printf(m, "%-25s %d\n", power_well->name,
power_well->count);
- for (power_domain = 0; power_domain < POWER_DOMAIN_NUM;
- power_domain++) {
- if (!(BIT(power_domain) & power_well->domains))
- continue;
-
+ for_each_power_domain(power_domain, power_well->domains)
seq_printf(m, " %-23s %d\n",
intel_display_power_domain_str(power_domain),
power_domains->domain_use_count[power_domain]);
- }
}
mutex_unlock(&power_domains->lock);
struct drm_device *dev = &dev_priv->drm;
struct intel_crtc *crtc;
struct drm_connector *connector;
+ struct drm_connector_list_iter conn_iter;
intel_runtime_pm_get(dev_priv);
- drm_modeset_lock_all(dev);
seq_printf(m, "CRTC info\n");
seq_printf(m, "---------\n");
for_each_intel_crtc(dev, crtc) {
struct intel_crtc_state *pipe_config;
int x, y;
+ drm_modeset_lock(&crtc->base.mutex, NULL);
pipe_config = to_intel_crtc_state(crtc->base.state);
seq_printf(m, "CRTC %d: pipe: %c, active=%s, (size=%dx%d), dither=%s, bpp=%d\n",
seq_printf(m, "\tunderrun reporting: cpu=%s pch=%s \n",
yesno(!crtc->cpu_fifo_underrun_disabled),
yesno(!crtc->pch_fifo_underrun_disabled));
+ drm_modeset_unlock(&crtc->base.mutex);
}
seq_printf(m, "\n");
seq_printf(m, "Connector info\n");
seq_printf(m, "--------------\n");
- list_for_each_entry(connector, &dev->mode_config.connector_list, head) {
+ mutex_lock(&dev->mode_config.mutex);
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_for_each_connector_iter(connector, &conn_iter)
intel_connector_info(m, connector);
- }
- drm_modeset_unlock_all(dev);
+ drm_connector_list_iter_end(&conn_iter);
+ mutex_unlock(&dev->mode_config.mutex);
+
intel_runtime_pm_put(dev_priv);
return 0;
intel_runtime_pm_get(dev_priv);
+ seq_printf(m, "GT awake? %s\n",
+ yesno(dev_priv->gt.awake));
+ seq_printf(m, "Global active requests: %d\n",
+ dev_priv->gt.active_requests);
+
for_each_engine(engine, dev_priv, id) {
struct intel_breadcrumbs *b = &engine->breadcrumbs;
struct drm_i915_gem_request *rq;
u64 addr;
seq_printf(m, "%s\n", engine->name);
- seq_printf(m, "\tcurrent seqno %x, last %x, hangcheck %x [%d ms]\n",
+ seq_printf(m, "\tcurrent seqno %x, last %x, hangcheck %x [%d ms], inflight %d\n",
intel_engine_get_seqno(engine),
intel_engine_last_submit(engine),
engine->hangcheck.seqno,
- jiffies_to_msecs(jiffies - engine->hangcheck.action_timestamp));
+ jiffies_to_msecs(jiffies - engine->hangcheck.action_timestamp),
+ engine->timeline->inflight_seqnos);
rcu_read_lock();
rcu_read_lock();
rq = READ_ONCE(engine->execlist_port[0].request);
- if (rq)
- print_request(m, rq, "\t\tELSP[0] ");
- else
+ if (rq) {
+ seq_printf(m, "\t\tELSP[0] count=%d, ",
+ engine->execlist_port[0].count);
+ print_request(m, rq, "rq: ");
+ } else {
seq_printf(m, "\t\tELSP[0] idle\n");
+ }
rq = READ_ONCE(engine->execlist_port[1].request);
- if (rq)
- print_request(m, rq, "\t\tELSP[1] ");
- else
+ if (rq) {
+ seq_printf(m, "\t\tELSP[1] count=%d, ",
+ engine->execlist_port[1].count);
+ print_request(m, rq, "rq: ");
+ } else {
seq_printf(m, "\t\tELSP[1] idle\n");
+ }
rcu_read_unlock();
spin_lock_irq(&engine->timeline->lock);
I915_READ(RING_PP_DIR_DCLV(engine)));
}
- spin_lock_irq(&b->lock);
+ spin_lock_irq(&b->rb_lock);
for (rb = rb_first(&b->waiters); rb; rb = rb_next(rb)) {
struct intel_wait *w = rb_entry(rb, typeof(*w), node);
seq_printf(m, "\t%s [%d] waiting for %x\n",
w->tsk->comm, w->tsk->pid, w->seqno);
}
- spin_unlock_irq(&b->lock);
+ spin_unlock_irq(&b->rb_lock);
seq_puts(m, "\n");
}
struct i915_drrs *drrs = &dev_priv->drrs;
int vrefresh = 0;
struct drm_connector *connector;
+ struct drm_connector_list_iter conn_iter;
- drm_for_each_connector(connector, dev) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->state->crtc != &intel_crtc->base)
continue;
seq_printf(m, "%s:\n", connector->name);
}
+ drm_connector_list_iter_end(&conn_iter);
if (dev_priv->vbt.drrs_type == STATIC_DRRS_SUPPORT)
seq_puts(m, "\tVBT: DRRS_type: Static");
struct intel_encoder *intel_encoder;
struct intel_digital_port *intel_dig_port;
struct drm_connector *connector;
+ struct drm_connector_list_iter conn_iter;
- drm_modeset_lock_all(dev);
- drm_for_each_connector(connector, dev) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->connector_type != DRM_MODE_CONNECTOR_DisplayPort)
continue;
port_name(intel_dig_port->port));
drm_dp_mst_dump_topology(m, &intel_dig_port->dp.mst_mgr);
}
- drm_modeset_unlock_all(dev);
+ drm_connector_list_iter_end(&conn_iter);
+
return 0;
}
int status = 0;
struct drm_device *dev;
struct drm_connector *connector;
- struct list_head *connector_list;
+ struct drm_connector_list_iter conn_iter;
struct intel_dp *intel_dp;
int val = 0;
dev = ((struct seq_file *)file->private_data)->private;
- connector_list = &dev->mode_config.connector_list;
-
if (len == 0)
return 0;
input_buffer[len] = '\0';
DRM_DEBUG_DRIVER("Copied %d bytes from user\n", (unsigned int)len);
- list_for_each_entry(connector, connector_list, head) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->connector_type !=
DRM_MODE_CONNECTOR_DisplayPort)
continue;
intel_dp = enc_to_intel_dp(connector->encoder);
status = kstrtoint(input_buffer, 10, &val);
if (status < 0)
- goto out;
+ break;
DRM_DEBUG_DRIVER("Got %d for test active\n", val);
/* To prevent erroneous activation of the compliance
* testing code, only accept an actual value of 1 here
intel_dp->compliance.test_active = 0;
}
}
+ drm_connector_list_iter_end(&conn_iter);
out:
kfree(input_buffer);
if (status < 0)
{
struct drm_device *dev = m->private;
struct drm_connector *connector;
- struct list_head *connector_list = &dev->mode_config.connector_list;
+ struct drm_connector_list_iter conn_iter;
struct intel_dp *intel_dp;
- list_for_each_entry(connector, connector_list, head) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->connector_type !=
DRM_MODE_CONNECTOR_DisplayPort)
continue;
} else
seq_puts(m, "0");
}
+ drm_connector_list_iter_end(&conn_iter);
return 0;
}
{
struct drm_device *dev = m->private;
struct drm_connector *connector;
- struct list_head *connector_list = &dev->mode_config.connector_list;
+ struct drm_connector_list_iter conn_iter;
struct intel_dp *intel_dp;
- list_for_each_entry(connector, connector_list, head) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->connector_type !=
DRM_MODE_CONNECTOR_DisplayPort)
continue;
if (connector->status == connector_status_connected &&
connector->encoder != NULL) {
intel_dp = enc_to_intel_dp(connector->encoder);
- seq_printf(m, "%lx", intel_dp->compliance.test_data.edid);
+ if (intel_dp->compliance.test_type ==
+ DP_TEST_LINK_EDID_READ)
+ seq_printf(m, "%lx",
+ intel_dp->compliance.test_data.edid);
+ else if (intel_dp->compliance.test_type ==
+ DP_TEST_LINK_VIDEO_PATTERN) {
+ seq_printf(m, "hdisplay: %d\n",
+ intel_dp->compliance.test_data.hdisplay);
+ seq_printf(m, "vdisplay: %d\n",
+ intel_dp->compliance.test_data.vdisplay);
+ seq_printf(m, "bpc: %u\n",
+ intel_dp->compliance.test_data.bpc);
+ }
} else
seq_puts(m, "0");
}
+ drm_connector_list_iter_end(&conn_iter);
return 0;
}
{
struct drm_device *dev = m->private;
struct drm_connector *connector;
- struct list_head *connector_list = &dev->mode_config.connector_list;
+ struct drm_connector_list_iter conn_iter;
struct intel_dp *intel_dp;
- list_for_each_entry(connector, connector_list, head) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->connector_type !=
DRM_MODE_CONNECTOR_DisplayPort)
continue;
} else
seq_puts(m, "0");
}
+ drm_connector_list_iter_end(&conn_iter);
return 0;
}
* while it is writing to 'i915_wedged'
*/
- if (i915_reset_in_progress(&dev_priv->gpu_error))
+ if (i915_reset_backoff(&dev_priv->gpu_error))
return -EAGAIN;
i915_handle_error(dev_priv, val,
"Manually setting wedged to %llu", val);
+ wait_on_bit(&dev_priv->gpu_error.flags,
+ I915_RESET_HANDOFF,
+ TASK_UNINTERRUPTIBLE);
+
return 0;
}
i915_wedged_get, i915_wedged_set,
"%llu\n");
+static int
+fault_irq_set(struct drm_i915_private *i915,
+ unsigned long *irq,
+ unsigned long val)
+{
+ int err;
+
+ err = mutex_lock_interruptible(&i915->drm.struct_mutex);
+ if (err)
+ return err;
+
+ err = i915_gem_wait_for_idle(i915,
+ I915_WAIT_LOCKED |
+ I915_WAIT_INTERRUPTIBLE);
+ if (err)
+ goto err_unlock;
+
+ /* Retire to kick idle work */
+ i915_gem_retire_requests(i915);
+ GEM_BUG_ON(i915->gt.active_requests);
+
+ *irq = val;
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ /* Flush idle worker to disarm irq */
+ while (flush_delayed_work(&i915->gt.idle_work))
+ ;
+
+ return 0;
+
+err_unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ return err;
+}
+
static int
i915_ring_missed_irq_get(void *data, u64 *val)
{
static int
i915_ring_missed_irq_set(void *data, u64 val)
{
- struct drm_i915_private *dev_priv = data;
- struct drm_device *dev = &dev_priv->drm;
- int ret;
+ struct drm_i915_private *i915 = data;
- /* Lock against concurrent debugfs callers */
- ret = mutex_lock_interruptible(&dev->struct_mutex);
- if (ret)
- return ret;
- dev_priv->gpu_error.missed_irq_rings = val;
- mutex_unlock(&dev->struct_mutex);
-
- return 0;
+ return fault_irq_set(i915, &i915->gpu_error.missed_irq_rings, val);
}
DEFINE_SIMPLE_ATTRIBUTE(i915_ring_missed_irq_fops,
static int
i915_ring_test_irq_set(void *data, u64 val)
{
- struct drm_i915_private *dev_priv = data;
+ struct drm_i915_private *i915 = data;
- val &= INTEL_INFO(dev_priv)->ring_mask;
+ val &= INTEL_INFO(i915)->ring_mask;
DRM_DEBUG_DRIVER("Masking interrupts on rings 0x%08llx\n", val);
- dev_priv->gpu_error.test_irq_rings = val;
- return 0;
+ return fault_irq_set(i915, &i915->gpu_error.test_irq_rings, val);
}
DEFINE_SIMPLE_ATTRIBUTE(i915_ring_test_irq_fops,
#define DROP_RETIRE 0x4
#define DROP_ACTIVE 0x8
#define DROP_FREED 0x10
+#define DROP_SHRINK_ALL 0x20
#define DROP_ALL (DROP_UNBOUND | \
DROP_BOUND | \
DROP_RETIRE | \
DROP_ACTIVE | \
- DROP_FREED)
+ DROP_FREED | \
+ DROP_SHRINK_ALL)
static int
i915_drop_caches_get(void *data, u64 *val)
{
if (val & (DROP_RETIRE | DROP_ACTIVE))
i915_gem_retire_requests(dev_priv);
+ lockdep_set_current_reclaim_state(GFP_KERNEL);
if (val & DROP_BOUND)
i915_gem_shrink(dev_priv, LONG_MAX, I915_SHRINK_BOUND);
if (val & DROP_UNBOUND)
i915_gem_shrink(dev_priv, LONG_MAX, I915_SHRINK_UNBOUND);
+ if (val & DROP_SHRINK_ALL)
+ i915_gem_shrink_all(dev_priv);
+ lockdep_clear_current_reclaim_state();
+
unlock:
mutex_unlock(&dev->struct_mutex);
dev_priv->rps.max_freq_softlimit = val;
- intel_set_rps(dev_priv, val);
+ if (intel_set_rps(dev_priv, val))
+ DRM_DEBUG_DRIVER("failed to update RPS to new softlimit\n");
mutex_unlock(&dev_priv->rps.hw_lock);
dev_priv->rps.min_freq_softlimit = val;
- intel_set_rps(dev_priv, val);
+ if (intel_set_rps(dev_priv, val))
+ DRM_DEBUG_DRIVER("failed to update RPS to new softlimit\n");
mutex_unlock(&dev_priv->rps.hw_lock);
sseu->slice_mask |= BIT(s);
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ if (IS_GEN9_BC(dev_priv))
sseu->subslice_mask =
INTEL_INFO(dev_priv)->sseu.subslice_mask;
.release = i915_forcewake_release,
};
-static int i915_forcewake_create(struct dentry *root, struct drm_minor *minor)
+static int i915_hpd_storm_ctl_show(struct seq_file *m, void *data)
{
- struct dentry *ent;
+ struct drm_i915_private *dev_priv = m->private;
+ struct i915_hotplug *hotplug = &dev_priv->hotplug;
- ent = debugfs_create_file("i915_forcewake_user",
- S_IRUSR,
- root, to_i915(minor->dev),
- &i915_forcewake_fops);
- if (!ent)
- return -ENOMEM;
+ seq_printf(m, "Threshold: %d\n", hotplug->hpd_storm_threshold);
+ seq_printf(m, "Detected: %s\n",
+ yesno(delayed_work_pending(&hotplug->reenable_work)));
- return drm_add_fake_info_node(minor, ent, &i915_forcewake_fops);
+ return 0;
}
-static int i915_debugfs_create(struct dentry *root,
- struct drm_minor *minor,
- const char *name,
- const struct file_operations *fops)
+static ssize_t i915_hpd_storm_ctl_write(struct file *file,
+ const char __user *ubuf, size_t len,
+ loff_t *offp)
{
- struct dentry *ent;
+ struct seq_file *m = file->private_data;
+ struct drm_i915_private *dev_priv = m->private;
+ struct i915_hotplug *hotplug = &dev_priv->hotplug;
+ unsigned int new_threshold;
+ int i;
+ char *newline;
+ char tmp[16];
- ent = debugfs_create_file(name,
- S_IRUGO | S_IWUSR,
- root, to_i915(minor->dev),
- fops);
- if (!ent)
- return -ENOMEM;
+ if (len >= sizeof(tmp))
+ return -EINVAL;
+
+ if (copy_from_user(tmp, ubuf, len))
+ return -EFAULT;
+
+ tmp[len] = '\0';
+
+ /* Strip newline, if any */
+ newline = strchr(tmp, '\n');
+ if (newline)
+ *newline = '\0';
+
+ if (strcmp(tmp, "reset") == 0)
+ new_threshold = HPD_STORM_DEFAULT_THRESHOLD;
+ else if (kstrtouint(tmp, 10, &new_threshold) != 0)
+ return -EINVAL;
+
+ if (new_threshold > 0)
+ DRM_DEBUG_KMS("Setting HPD storm detection threshold to %d\n",
+ new_threshold);
+ else
+ DRM_DEBUG_KMS("Disabling HPD storm detection\n");
- return drm_add_fake_info_node(minor, ent, fops);
+ spin_lock_irq(&dev_priv->irq_lock);
+ hotplug->hpd_storm_threshold = new_threshold;
+ /* Reset the HPD storm stats so we don't accidentally trigger a storm */
+ for_each_hpd_pin(i)
+ hotplug->stats[i].count = 0;
+ spin_unlock_irq(&dev_priv->irq_lock);
+
+ /* Re-enable hpd immediately if we were in an irq storm */
+ flush_delayed_work(&dev_priv->hotplug.reenable_work);
+
+ return len;
+}
+
+static int i915_hpd_storm_ctl_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, i915_hpd_storm_ctl_show, inode->i_private);
}
+static const struct file_operations i915_hpd_storm_ctl_fops = {
+ .owner = THIS_MODULE,
+ .open = i915_hpd_storm_ctl_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+ .write = i915_hpd_storm_ctl_write
+};
+
static const struct drm_info_list i915_debugfs_list[] = {
{"i915_capabilities", i915_capabilities, 0},
{"i915_gem_objects", i915_gem_object_info, 0},
{"i915_gem_drop_caches", &i915_drop_caches_fops},
#if IS_ENABLED(CONFIG_DRM_I915_CAPTURE_ERROR)
{"i915_error_state", &i915_error_state_fops},
+ {"i915_gpu_info", &i915_gpu_info_fops},
#endif
{"i915_next_seqno", &i915_next_seqno_fops},
{"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
{"i915_dp_test_data", &i915_displayport_test_data_fops},
{"i915_dp_test_type", &i915_displayport_test_type_fops},
{"i915_dp_test_active", &i915_displayport_test_active_fops},
- {"i915_guc_log_control", &i915_guc_log_control_fops}
+ {"i915_guc_log_control", &i915_guc_log_control_fops},
+ {"i915_hpd_storm_ctl", &i915_hpd_storm_ctl_fops}
};
int i915_debugfs_register(struct drm_i915_private *dev_priv)
{
struct drm_minor *minor = dev_priv->drm.primary;
+ struct dentry *ent;
int ret, i;
- ret = i915_forcewake_create(minor->debugfs_root, minor);
- if (ret)
- return ret;
+ ent = debugfs_create_file("i915_forcewake_user", S_IRUSR,
+ minor->debugfs_root, to_i915(minor->dev),
+ &i915_forcewake_fops);
+ if (!ent)
+ return -ENOMEM;
ret = intel_pipe_crc_create(minor);
if (ret)
return ret;
for (i = 0; i < ARRAY_SIZE(i915_debugfs_files); i++) {
- ret = i915_debugfs_create(minor->debugfs_root, minor,
- i915_debugfs_files[i].name,
+ ent = debugfs_create_file(i915_debugfs_files[i].name,
+ S_IRUGO | S_IWUSR,
+ minor->debugfs_root,
+ to_i915(minor->dev),
i915_debugfs_files[i].fops);
- if (ret)
- return ret;
+ if (!ent)
+ return -ENOMEM;
}
return drm_debugfs_create_files(i915_debugfs_list,
minor->debugfs_root, minor);
}
-void i915_debugfs_unregister(struct drm_i915_private *dev_priv)
-{
- struct drm_minor *minor = dev_priv->drm.primary;
- int i;
-
- drm_debugfs_remove_files(i915_debugfs_list,
- I915_DEBUGFS_ENTRIES, minor);
-
- drm_debugfs_remove_files((struct drm_info_list *)&i915_forcewake_fops,
- 1, minor);
-
- intel_pipe_crc_cleanup(minor);
-
- for (i = 0; i < ARRAY_SIZE(i915_debugfs_files); i++) {
- struct drm_info_list *info_list =
- (struct drm_info_list *)i915_debugfs_files[i].fops;
-
- drm_debugfs_remove_files(info_list, 1, minor);
- }
-}
-
struct dpcd_block {
/* DPCD dump start address. */
unsigned int offset;
#include <drm/drmP.h>
#include <drm/drm_crtc_helper.h>
+#include <drm/drm_atomic_helper.h>
#include <drm/i915_drm.h>
#include "i915_drv.h"
value = INTEL_INFO(dev_priv)->sseu.min_eu_in_pool;
break;
case I915_PARAM_HUC_STATUS:
- /* The register is already force-woken. We dont need
- * any rpm here
- */
+ intel_runtime_pm_get(dev_priv);
value = I915_READ(HUC_STATUS2) & HUC_FW_VERIFIED;
+ intel_runtime_pm_put(dev_priv);
break;
case I915_PARAM_MMAP_GTT_VERSION:
/* Though we've started our numbering from 1, and so class all
case I915_PARAM_HAS_EXEC_HANDLE_LUT:
case I915_PARAM_HAS_COHERENT_PHYS_GTT:
case I915_PARAM_HAS_EXEC_SOFTPIN:
+ case I915_PARAM_HAS_EXEC_ASYNC:
+ case I915_PARAM_HAS_EXEC_FENCE:
/* For the time being all of these are always true;
* if some supported hardware does not have one of these
* features this value needs to be provided from
if (i915_inject_load_failure())
return -ENODEV;
- ret = intel_bios_init(dev_priv);
- if (ret)
- DRM_INFO("failed to find VBIOS tables\n");
+ intel_bios_init(dev_priv);
/* If we have > 1 VGA cards, then we need to arbitrate access
* to the common VGA resources.
if (ret)
goto cleanup_irq;
- intel_huc_init(dev_priv);
- intel_guc_init(dev_priv);
+ intel_uc_init_fw(dev_priv);
ret = i915_gem_init(dev_priv);
if (ret)
return -ENOMEM;
}
+static void i915_engines_cleanup(struct drm_i915_private *i915)
+{
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+
+ for_each_engine(engine, i915, id)
+ kfree(engine);
+}
+
static void i915_workqueues_cleanup(struct drm_i915_private *dev_priv)
{
destroy_workqueue(dev_priv->hotplug.dp_wq);
*/
static void intel_detect_preproduction_hw(struct drm_i915_private *dev_priv)
{
- if (IS_HSW_EARLY_SDV(dev_priv) ||
- IS_SKL_REVID(dev_priv, 0, SKL_REVID_F0))
+ bool pre = false;
+
+ pre |= IS_HSW_EARLY_SDV(dev_priv);
+ pre |= IS_SKL_REVID(dev_priv, 0, SKL_REVID_F0);
+ pre |= IS_BXT_REVID(dev_priv, 0, BXT_REVID_B_LAST);
+
+ if (pre) {
DRM_ERROR("This is a pre-production stepping. "
"It may not be fully functional.\n");
+ add_taint(TAINT_MACHINE_CHECK, LOCKDEP_STILL_OK);
+ }
}
/**
spin_lock_init(&dev_priv->gpu_error.lock);
mutex_init(&dev_priv->backlight_lock);
spin_lock_init(&dev_priv->uncore.lock);
+
spin_lock_init(&dev_priv->mm.object_stat_lock);
spin_lock_init(&dev_priv->mmio_flip_lock);
- spin_lock_init(&dev_priv->wm.dsparb_lock);
mutex_init(&dev_priv->sb_lock);
mutex_init(&dev_priv->modeset_restore_lock);
mutex_init(&dev_priv->av_mutex);
mutex_init(&dev_priv->pps_mutex);
intel_uc_init_early(dev_priv);
-
i915_memcpy_init_early(dev_priv);
+ ret = intel_engines_init_early(dev_priv);
+ if (ret)
+ return ret;
+
ret = i915_workqueues_init(dev_priv);
if (ret < 0)
- return ret;
+ goto err_engines;
/* This must be called before any calls to HAS_PCH_* */
intel_detect_pch(dev_priv);
err_workqueues:
i915_workqueues_cleanup(dev_priv);
+err_engines:
+ i915_engines_cleanup(dev_priv);
return ret;
}
i915_perf_fini(dev_priv);
i915_gem_load_cleanup(dev_priv);
i915_workqueues_cleanup(dev_priv);
+ i915_engines_cleanup(dev_priv);
}
static int i915_mmio_setup(struct drm_i915_private *dev_priv)
goto put_bridge;
intel_uncore_init(dev_priv);
+ i915_gem_init_mmio(dev_priv);
return 0;
DRM_DEBUG_DRIVER("ppgtt mode: %i\n", i915.enable_ppgtt);
i915.semaphores = intel_sanitize_semaphores(dev_priv, i915.semaphores);
- DRM_DEBUG_DRIVER("use GPU sempahores? %s\n", yesno(i915.semaphores));
+ DRM_DEBUG_DRIVER("use GPU semaphores? %s\n", yesno(i915.semaphores));
+
+ intel_uc_sanitize_options(dev_priv);
}
/**
i915_teardown_sysfs(dev_priv);
i915_guc_log_unregister(dev_priv);
- i915_debugfs_unregister(dev_priv);
drm_dev_unregister(&dev_priv->drm);
i915_gem_shrinker_cleanup(dev_priv);
*/
int i915_driver_load(struct pci_dev *pdev, const struct pci_device_id *ent)
{
+ const struct intel_device_info *match_info =
+ (struct intel_device_info *)ent->driver_data;
struct drm_i915_private *dev_priv;
int ret;
- if (i915.nuclear_pageflip)
- driver.driver_features |= DRIVER_ATOMIC;
+ /* Enable nuclear pageflip on ILK+, except vlv/chv */
+ if (!i915.nuclear_pageflip &&
+ (match_info->gen < 5 || match_info->has_gmch_display))
+ driver.driver_features &= ~DRIVER_ATOMIC;
ret = -ENOMEM;
dev_priv = kzalloc(sizeof(*dev_priv), GFP_KERNEL);
ret = drm_dev_init(&dev_priv->drm, &driver, &pdev->dev);
if (ret) {
DRM_DEV_ERROR(&pdev->dev, "allocation failed\n");
- kfree(dev_priv);
- return ret;
+ goto out_free;
}
dev_priv->drm.pdev = pdev;
ret = pci_enable_device(pdev);
if (ret)
- goto out_free_priv;
+ goto out_fini;
pci_set_drvdata(pdev, &dev_priv->drm);
i915_driver_cleanup_early(dev_priv);
out_pci_disable:
pci_disable_device(pdev);
-out_free_priv:
+out_fini:
i915_load_error(dev_priv, "Device initialization failed (%d)\n", ret);
- drm_dev_unref(&dev_priv->drm);
+ drm_dev_fini(&dev_priv->drm);
+out_free:
+ kfree(dev_priv);
return ret;
}
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct pci_dev *pdev = dev_priv->drm.pdev;
+ struct drm_modeset_acquire_ctx ctx;
+ int ret;
intel_fbdev_fini(dev);
intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
+ drm_modeset_acquire_init(&ctx, 0);
+ while (1) {
+ ret = drm_modeset_lock_all_ctx(dev, &ctx);
+ if (!ret)
+ ret = drm_atomic_helper_disable_all(dev, &ctx);
+
+ if (ret != -EDEADLK)
+ break;
+
+ drm_modeset_backoff(&ctx);
+ }
+
+ if (ret)
+ DRM_ERROR("Disabling all crtc's during unload failed with %i\n", ret);
+
+ drm_modeset_drop_locks(&ctx);
+ drm_modeset_acquire_fini(&ctx);
+
intel_gvt_cleanup(dev_priv);
i915_driver_unregister(dev_priv);
/* Free error state after interrupts are fully disabled. */
cancel_delayed_work_sync(&dev_priv->gpu_error.hangcheck_work);
- i915_destroy_error_state(dev_priv);
+ i915_reset_error_state(dev_priv);
/* Flush any outstanding unpin_work. */
drain_workqueue(dev_priv->wq);
i915_driver_cleanup_mmio(dev_priv);
intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
+}
+
+static void i915_driver_release(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
i915_driver_cleanup_early(dev_priv);
+ drm_dev_fini(&dev_priv->drm);
+
+ kfree(dev_priv);
}
static int i915_driver_open(struct drm_device *dev, struct drm_file *file)
vga_switcheroo_process_delayed_switch();
}
-static void i915_driver_preclose(struct drm_device *dev, struct drm_file *file)
+static void i915_driver_postclose(struct drm_device *dev, struct drm_file *file)
{
+ struct drm_i915_file_private *file_priv = file->driver_priv;
+
mutex_lock(&dev->struct_mutex);
i915_gem_context_close(dev, file);
i915_gem_release(dev, file);
mutex_unlock(&dev->struct_mutex);
-}
-
-static void i915_driver_postclose(struct drm_device *dev, struct drm_file *file)
-{
- struct drm_i915_file_private *file_priv = file->driver_priv;
kfree(file_priv);
}
opregion_target_state = suspend_to_idle(dev_priv) ? PCI_D1 : PCI_D3cold;
intel_opregion_notify_adapter(dev_priv, opregion_target_state);
- intel_uncore_forcewake_reset(dev_priv, false);
+ intel_uncore_suspend(dev_priv);
intel_opregion_unregister(dev_priv);
intel_fbdev_set_suspend(dev, FBINFO_STATE_SUSPENDED, true);
DRM_ERROR("Resume prepare failed: %d, continuing anyway\n",
ret);
- intel_uncore_early_sanitize(dev_priv, true);
+ intel_uncore_resume_early(dev_priv);
if (IS_GEN9_LP(dev_priv)) {
if (!dev_priv->suspended_to_idle)
!(dev_priv->suspended_to_idle && dev_priv->csr.dmc_payload))
intel_power_domains_init_hw(dev_priv, true);
+ i915_gem_sanitize(dev_priv);
+
enable_rpm_wakeref_asserts(dev_priv);
out:
int ret;
lockdep_assert_held(&dev_priv->drm.struct_mutex);
+ GEM_BUG_ON(!test_bit(I915_RESET_BACKOFF, &error->flags));
- if (!test_and_clear_bit(I915_RESET_IN_PROGRESS, &error->flags))
+ if (!test_bit(I915_RESET_HANDOFF, &error->flags))
return;
/* Clear any previous failed attempts at recovery. Time to try again. */
- __clear_bit(I915_WEDGED, &error->flags);
+ if (!i915_gem_unset_wedged(dev_priv))
+ goto wakeup;
+
error->reset_count++;
pr_notice("drm/i915: Resetting chip after gpu hang\n");
i915_queue_hangcheck(dev_priv);
-wakeup:
+finish:
i915_gem_reset_finish(dev_priv);
enable_irq(dev_priv->drm.irq);
- wake_up_bit(&error->flags, I915_RESET_IN_PROGRESS);
+
+wakeup:
+ clear_bit(I915_RESET_HANDOFF, &error->flags);
+ wake_up_bit(&error->flags, I915_RESET_HANDOFF);
return;
error:
i915_gem_set_wedged(dev_priv);
- goto wakeup;
+ goto finish;
}
static int i915_pm_suspend(struct device *kdev)
return ret;
}
- intel_uncore_forcewake_reset(dev_priv, false);
+ intel_uncore_suspend(dev_priv);
enable_rpm_wakeref_asserts(dev_priv);
WARN_ON_ONCE(atomic_read(&dev_priv->pm.wakeref_count));
DRM_IOCTL_DEF_DRV(I915_HWS_ADDR, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_GEM_INIT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_GEM_EXECBUFFER, i915_gem_execbuffer, DRM_AUTH),
- DRM_IOCTL_DEF_DRV(I915_GEM_EXECBUFFER2, i915_gem_execbuffer2, DRM_AUTH|DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_EXECBUFFER2_WR, i915_gem_execbuffer2, DRM_AUTH|DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_PIN, i915_gem_reject_pin_ioctl, DRM_AUTH|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_GEM_UNPIN, i915_gem_reject_pin_ioctl, DRM_AUTH|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_GEM_BUSY, i915_gem_busy_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
*/
.driver_features =
DRIVER_HAVE_IRQ | DRIVER_IRQ_SHARED | DRIVER_GEM | DRIVER_PRIME |
- DRIVER_RENDER | DRIVER_MODESET,
+ DRIVER_RENDER | DRIVER_MODESET | DRIVER_ATOMIC,
+ .release = i915_driver_release,
.open = i915_driver_open,
.lastclose = i915_driver_lastclose,
- .preclose = i915_driver_preclose,
.postclose = i915_driver_postclose,
.set_busid = drm_pci_set_busid,
.minor = DRIVER_MINOR,
.patchlevel = DRIVER_PATCHLEVEL,
};
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/mock_drm.c"
+#endif
#define DRIVER_NAME "i915"
#define DRIVER_DESC "Intel Graphics"
-#define DRIVER_DATE "20170123"
-#define DRIVER_TIMESTAMP 1485156432
+#define DRIVER_DATE "20170320"
+#define DRIVER_TIMESTAMP 1489994464
#undef WARN_ON
/* Many gcc seem to no see through this and fall over :( */
POWER_DOMAIN_PORT_DDI_C_LANES,
POWER_DOMAIN_PORT_DDI_D_LANES,
POWER_DOMAIN_PORT_DDI_E_LANES,
+ POWER_DOMAIN_PORT_DDI_A_IO,
+ POWER_DOMAIN_PORT_DDI_B_IO,
+ POWER_DOMAIN_PORT_DDI_C_IO,
+ POWER_DOMAIN_PORT_DDI_D_IO,
+ POWER_DOMAIN_PORT_DDI_E_IO,
POWER_DOMAIN_PORT_DSI,
POWER_DOMAIN_PORT_CRT,
POWER_DOMAIN_PORT_OTHER,
#define for_each_hpd_pin(__pin) \
for ((__pin) = (HPD_NONE + 1); (__pin) < HPD_NUM_PINS; (__pin)++)
+#define HPD_STORM_DEFAULT_THRESHOLD 5
+
struct i915_hotplug {
struct work_struct hotplug_work;
struct work_struct poll_init_work;
bool poll_enabled;
+ unsigned int hpd_storm_threshold;
+
/*
* if we get a HPD irq from DP and a HPD irq from non-DP
* the non-DP HPD could block the workqueue on a mode config
&(dev)->mode_config.encoder_list, \
base.head)
-#define for_each_intel_connector(dev, intel_connector) \
- list_for_each_entry(intel_connector, \
- &(dev)->mode_config.connector_list, \
- base.head)
+#define for_each_intel_connector_iter(intel_connector, iter) \
+ while ((intel_connector = to_intel_connector(drm_connector_list_iter_next(iter))))
#define for_each_encoder_on_crtc(dev, __crtc, intel_encoder) \
list_for_each_entry((intel_encoder), &(dev)->mode_config.encoder_list, base.head) \
#define for_each_power_domain(domain, mask) \
for ((domain) = 0; (domain) < POWER_DOMAIN_NUM; (domain)++) \
- for_each_if ((1 << (domain)) & (mask))
+ for_each_if (BIT_ULL(domain) & (mask))
+
+#define for_each_power_well(__dev_priv, __power_well) \
+ for ((__power_well) = (__dev_priv)->power_domains.power_wells; \
+ (__power_well) - (__dev_priv)->power_domains.power_wells < \
+ (__dev_priv)->power_domains.power_well_count; \
+ (__power_well)++)
+
+#define for_each_power_well_rev(__dev_priv, __power_well) \
+ for ((__power_well) = (__dev_priv)->power_domains.power_wells + \
+ (__dev_priv)->power_domains.power_well_count - 1; \
+ (__power_well) - (__dev_priv)->power_domains.power_wells >= 0; \
+ (__power_well)--)
+
+#define for_each_power_domain_well(__dev_priv, __power_well, __domain_mask) \
+ for_each_power_well(__dev_priv, __power_well) \
+ for_each_if ((__power_well)->domains & (__domain_mask))
+
+#define for_each_power_domain_well_rev(__dev_priv, __power_well, __domain_mask) \
+ for_each_power_well_rev(__dev_priv, __power_well) \
+ for_each_if ((__power_well)->domains & (__domain_mask))
+
+#define for_each_intel_plane_in_state(__state, plane, plane_state, __i) \
+ for ((__i) = 0; \
+ (__i) < (__state)->base.dev->mode_config.num_total_plane && \
+ ((plane) = to_intel_plane((__state)->base.planes[__i].ptr), \
+ (plane_state) = to_intel_plane_state((__state)->base.planes[__i].state), 1); \
+ (__i)++) \
+ for_each_if (plane_state)
struct drm_i915_private;
struct i915_mm_struct;
struct intel_crtc;
struct intel_limit;
struct dpll;
+struct intel_cdclk_state;
struct drm_i915_display_funcs {
- int (*get_display_clock_speed)(struct drm_i915_private *dev_priv);
+ void (*get_cdclk)(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state);
+ void (*set_cdclk)(struct drm_i915_private *dev_priv,
+ const struct intel_cdclk_state *cdclk_state);
int (*get_fifo_size)(struct drm_i915_private *dev_priv, int plane);
int (*compute_pipe_wm)(struct intel_crtc_state *cstate);
int (*compute_intermediate_wm)(struct drm_device *dev,
int (*compute_global_watermarks)(struct drm_atomic_state *state);
void (*update_wm)(struct intel_crtc *crtc);
int (*modeset_calc_cdclk)(struct drm_atomic_state *state);
- void (*modeset_commit_cdclk)(struct drm_atomic_state *state);
/* Returns the active state of the crtc, and if the crtc is active,
* fills out the pipe-config with the hw state. */
bool (*get_pipe_config)(struct intel_crtc *,
struct intel_encoder *encoder,
const struct drm_display_mode *adjusted_mode);
void (*audio_codec_disable)(struct intel_encoder *encoder);
- void (*fdi_link_train)(struct drm_crtc *crtc);
+ void (*fdi_link_train)(struct intel_crtc *crtc,
+ const struct intel_crtc_state *crtc_state);
void (*init_clock_gating)(struct drm_i915_private *dev_priv);
int (*queue_flip)(struct drm_device *dev, struct drm_crtc *crtc,
struct drm_framebuffer *fb,
const struct intel_forcewake_range *fw_domains_table;
unsigned int fw_domains_table_entries;
+ struct notifier_block pmic_bus_access_nb;
struct intel_uncore_funcs funcs;
unsigned fifo_count;
INTEL_BROXTON,
INTEL_KABYLAKE,
INTEL_GEMINILAKE,
+ INTEL_MAX_PLATFORMS
};
struct intel_device_info {
struct intel_display_error_state;
-struct drm_i915_error_state {
+struct i915_gpu_state {
struct kref ref;
struct timeval time;
struct timeval boottime;
char error_msg[128];
bool simulated;
+ bool awake;
+ bool wakelock;
+ bool suspended;
int iommu;
u32 reset_count;
u32 suspend_count;
struct intel_device_info device_info;
+ struct i915_params params;
/* Generic register state */
u32 eir;
u32 pgtbl_er;
u32 ier;
- u32 gtier[4];
+ u32 gtier[4], ngtier;
u32 ccid;
u32 derrmr;
u32 forcewake;
u32 gab_ctl;
u32 gfx_mode;
+ u32 nfence;
u64 fence[I915_MAX_NUM_FENCES];
struct intel_overlay_error_state *overlay;
struct intel_display_error_state *display;
u32 semaphore_mboxes[I915_NUM_ENGINES - 1];
struct intel_instdone instdone;
+ struct drm_i915_error_context {
+ char comm[TASK_COMM_LEN];
+ pid_t pid;
+ u32 handle;
+ u32 hw_id;
+ int ban_score;
+ int active;
+ int guilty;
+ } context;
+
struct drm_i915_error_object {
u64 gtt_offset;
u64 gtt_size;
u32 pp_dir_base;
};
} vm_info;
-
- pid_t pid;
- char comm[TASK_COMM_LEN];
- int context_bans;
} engine[I915_NUM_ENGINES];
struct drm_i915_error_buffer {
};
struct intel_rps_ei {
- u32 cz_clock;
+ ktime_t ktime;
u32 render_c0;
u32 media_c0;
};
u32 pm_iir;
/* PM interrupt bits that should never be masked */
- u32 pm_intr_keep;
+ u32 pm_intrmsk_mbz;
/* Frequencies are stored in potentially platform dependent multiples.
* In other words, *_freq needs to be multiplied by X to be interesting.
int count;
/* cached hw enabled state */
bool hw_enabled;
- unsigned long domains;
+ u64 domains;
/* unique identifier for this power well */
unsigned long id;
/*
struct work_struct free_work;
/** Usable portion of the GTT for GEM */
- phys_addr_t stolen_base; /* limited to low memory (32-bit) */
+ dma_addr_t stolen_base; /* limited to low memory (32-bit) */
/** PPGTT used for aliasing the PPGTT with the GTT */
struct i915_hw_ppgtt *aliasing_ppgtt;
loff_t pos;
};
-struct i915_error_state_file_priv {
- struct drm_i915_private *i915;
- struct drm_i915_error_state *error;
-};
-
#define I915_RESET_TIMEOUT (10 * HZ) /* 10s */
#define I915_FENCE_TIMEOUT (10 * HZ) /* 10s */
/* For reset and error_state handling. */
spinlock_t lock;
/* Protected by the above dev->gpu_error.lock. */
- struct drm_i915_error_state *first_error;
+ struct i915_gpu_state *first_error;
unsigned long missed_irq_rings;
*/
unsigned long reset_count;
+ /**
+ * flags: Control various stages of the GPU reset
+ *
+ * #I915_RESET_BACKOFF - When we start a reset, we want to stop any
+ * other users acquiring the struct_mutex. To do this we set the
+ * #I915_RESET_BACKOFF bit in the error flags when we detect a reset
+ * and then check for that bit before acquiring the struct_mutex (in
+ * i915_mutex_lock_interruptible()?). I915_RESET_BACKOFF serves a
+ * secondary role in preventing two concurrent global reset attempts.
+ *
+ * #I915_RESET_HANDOFF - To perform the actual GPU reset, we need the
+ * struct_mutex. We try to acquire the struct_mutex in the reset worker,
+ * but it may be held by some long running waiter (that we cannot
+ * interrupt without causing trouble). Once we are ready to do the GPU
+ * reset, we set the I915_RESET_HANDOFF bit and wakeup any waiters. If
+ * they already hold the struct_mutex and want to participate they can
+ * inspect the bit and do the reset directly, otherwise the worker
+ * waits for the struct_mutex.
+ *
+ * #I915_WEDGED - If reset fails and we can no longer use the GPU,
+ * we set the #I915_WEDGED bit. Prior to command submission, e.g.
+ * i915_gem_request_alloc(), this bit is checked and the sequence
+ * aborted (with -EIO reported to userspace) if set.
+ */
unsigned long flags;
-#define I915_RESET_IN_PROGRESS 0
+#define I915_RESET_BACKOFF 0
+#define I915_RESET_HANDOFF 1
#define I915_WEDGED (BITS_PER_LONG - 1)
/**
bool (*oa_buffer_is_empty)(struct drm_i915_private *dev_priv);
};
+struct intel_cdclk_state {
+ unsigned int cdclk, vco, ref;
+};
+
struct drm_i915_private {
struct drm_device drm;
unsigned int fsb_freq, mem_freq, is_ddr3;
unsigned int skl_preferred_vco_freq;
- unsigned int cdclk_freq, max_cdclk_freq;
-
- /*
- * For reading holding any crtc lock is sufficient,
- * for writing must hold all of them.
- */
- unsigned int atomic_cdclk_freq;
+ unsigned int max_cdclk_freq;
unsigned int max_dotclk_freq;
unsigned int rawclk_freq;
unsigned int czclk_freq;
struct {
- unsigned int vco, ref;
- } cdclk_pll;
+ /*
+ * The current logical cdclk state.
+ * See intel_atomic_state.cdclk.logical
+ *
+ * For reading holding any crtc lock is sufficient,
+ * for writing must hold all of them.
+ */
+ struct intel_cdclk_state logical;
+ /*
+ * The current actual cdclk state.
+ * See intel_atomic_state.cdclk.actual
+ */
+ struct intel_cdclk_state actual;
+ /* The current hardware cdclk state */
+ struct intel_cdclk_state hw;
+ } cdclk;
/**
* wq - Driver workqueue for GEM.
} sagv_status;
struct {
- /* protects DSPARB registers on pre-g4x/vlv/chv */
- spinlock_t dsparb_lock;
-
/*
* Raw watermark latency values:
* in 0.1us units for WM0,
return container_of(guc, struct drm_i915_private, guc);
}
+static inline struct drm_i915_private *huc_to_i915(struct intel_huc *huc)
+{
+ return container_of(huc, struct drm_i915_private, huc);
+}
+
/* Simple iterator over all initialised engines */
#define for_each_engine(engine__, dev_priv__, id__) \
for ((id__) = 0; \
#define IS_KBL_REVID(dev_priv, since, until) \
(IS_KABYLAKE(dev_priv) && IS_REVID(dev_priv, since, until))
+#define GLK_REVID_A0 0x0
+#define GLK_REVID_A1 0x1
+
+#define IS_GLK_REVID(dev_priv, since, until) \
+ (IS_GEMINILAKE(dev_priv) && IS_REVID(dev_priv, since, until))
+
/*
* The genX designation typically refers to the render engine, so render
* capability related checks should use IS_GEN, while display and other checks
#define IS_GEN8(dev_priv) (!!((dev_priv)->info.gen_mask & BIT(7)))
#define IS_GEN9(dev_priv) (!!((dev_priv)->info.gen_mask & BIT(8)))
-#define IS_GEN9_LP(dev_priv) (IS_GEN9(dev_priv) && INTEL_INFO(dev_priv)->is_lp)
#define IS_LP(dev_priv) (INTEL_INFO(dev_priv)->is_lp)
+#define IS_GEN9_LP(dev_priv) (IS_GEN9(dev_priv) && IS_LP(dev_priv))
+#define IS_GEN9_BC(dev_priv) (IS_GEN9(dev_priv) && !IS_LP(dev_priv))
#define ENGINE_MASK(id) BIT(id)
#define RENDER_RING ENGINE_MASK(RCS)
/* WaRsDisableCoarsePowerGating:skl,bxt */
#define NEEDS_WaRsDisableCoarsePowerGating(dev_priv) \
- (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1) || \
- IS_SKL_GT3(dev_priv) || \
- IS_SKL_GT4(dev_priv))
+ (IS_SKL_GT3(dev_priv) || IS_SKL_GT4(dev_priv))
/*
* dp aux and gmbus irq on gen4 seems to be able to generate legacy interrupts
extern void i915_update_gfx_val(struct drm_i915_private *dev_priv);
int vlv_force_gfx_clock(struct drm_i915_private *dev_priv, bool on);
+int intel_engines_init_early(struct drm_i915_private *dev_priv);
+int intel_engines_init(struct drm_i915_private *dev_priv);
+
/* intel_hotplug.c */
void intel_hpd_irq_handler(struct drm_i915_private *dev_priv,
u32 pin_mask, u32 long_mask);
void intel_irq_uninstall(struct drm_i915_private *dev_priv);
extern void intel_uncore_sanitize(struct drm_i915_private *dev_priv);
-extern void intel_uncore_early_sanitize(struct drm_i915_private *dev_priv,
- bool restore_forcewake);
extern void intel_uncore_init(struct drm_i915_private *dev_priv);
extern bool intel_uncore_unclaimed_mmio(struct drm_i915_private *dev_priv);
extern bool intel_uncore_arm_unclaimed_mmio_detection(struct drm_i915_private *dev_priv);
extern void intel_uncore_fini(struct drm_i915_private *dev_priv);
-extern void intel_uncore_forcewake_reset(struct drm_i915_private *dev_priv,
- bool restore);
+extern void intel_uncore_suspend(struct drm_i915_private *dev_priv);
+extern void intel_uncore_resume_early(struct drm_i915_private *dev_priv);
const char *intel_uncore_forcewake_domain_to_str(const enum forcewake_domain_id id);
void intel_uncore_forcewake_get(struct drm_i915_private *dev_priv,
enum forcewake_domains domains);
struct drm_file *file_priv);
int i915_gem_wait_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
+void i915_gem_sanitize(struct drm_i915_private *i915);
int i915_gem_load_init(struct drm_i915_private *dev_priv);
void i915_gem_load_cleanup(struct drm_i915_private *dev_priv);
void i915_gem_load_init_fences(struct drm_i915_private *dev_priv);
unsigned int *needs_clflush);
int i915_gem_obj_prepare_shmem_write(struct drm_i915_gem_object *obj,
unsigned int *needs_clflush);
-#define CLFLUSH_BEFORE 0x1
-#define CLFLUSH_AFTER 0x2
-#define CLFLUSH_FLAGS (CLFLUSH_BEFORE | CLFLUSH_AFTER)
+#define CLFLUSH_BEFORE BIT(0)
+#define CLFLUSH_AFTER BIT(1)
+#define CLFLUSH_FLAGS (CLFLUSH_BEFORE | CLFLUSH_AFTER)
static inline void
i915_gem_obj_finish_shmem_access(struct drm_i915_gem_object *obj)
void i915_gem_retire_requests(struct drm_i915_private *dev_priv);
-static inline bool i915_reset_in_progress(struct i915_gpu_error *error)
+static inline bool i915_reset_backoff(struct i915_gpu_error *error)
+{
+ return unlikely(test_bit(I915_RESET_BACKOFF, &error->flags));
+}
+
+static inline bool i915_reset_handoff(struct i915_gpu_error *error)
{
- return unlikely(test_bit(I915_RESET_IN_PROGRESS, &error->flags));
+ return unlikely(test_bit(I915_RESET_HANDOFF, &error->flags));
}
static inline bool i915_terminally_wedged(struct i915_gpu_error *error)
return unlikely(test_bit(I915_WEDGED, &error->flags));
}
-static inline bool i915_reset_in_progress_or_wedged(struct i915_gpu_error *error)
+static inline bool i915_reset_backoff_or_wedged(struct i915_gpu_error *error)
{
- return i915_reset_in_progress(error) | i915_terminally_wedged(error);
+ return i915_reset_backoff(error) | i915_terminally_wedged(error);
}
static inline u32 i915_reset_count(struct i915_gpu_error *error)
void i915_gem_reset(struct drm_i915_private *dev_priv);
void i915_gem_reset_finish(struct drm_i915_private *dev_priv);
void i915_gem_set_wedged(struct drm_i915_private *dev_priv);
-void i915_gem_clflush_object(struct drm_i915_gem_object *obj, bool force);
+bool i915_gem_unset_wedged(struct drm_i915_private *dev_priv);
+
+void i915_gem_init_mmio(struct drm_i915_private *i915);
int __must_check i915_gem_init(struct drm_i915_private *dev_priv);
int __must_check i915_gem_init_hw(struct drm_i915_private *dev_priv);
void i915_gem_init_swizzling(struct drm_i915_private *dev_priv);
void i915_gem_cleanup_engines(struct drm_i915_private *dev_priv);
-int __must_check i915_gem_wait_for_idle(struct drm_i915_private *dev_priv,
- unsigned int flags);
+int i915_gem_wait_for_idle(struct drm_i915_private *dev_priv,
+ unsigned int flags);
int __must_check i915_gem_suspend(struct drm_i915_private *dev_priv);
void i915_gem_resume(struct drm_i915_private *dev_priv);
int i915_gem_fault(struct vm_fault *vmf);
/* i915_debugfs.c */
#ifdef CONFIG_DEBUG_FS
int i915_debugfs_register(struct drm_i915_private *dev_priv);
-void i915_debugfs_unregister(struct drm_i915_private *dev_priv);
int i915_debugfs_connector_add(struct drm_connector *connector);
void intel_display_crc_init(struct drm_i915_private *dev_priv);
#else
static inline int i915_debugfs_register(struct drm_i915_private *dev_priv) {return 0;}
-static inline void i915_debugfs_unregister(struct drm_i915_private *dev_priv) {}
static inline int i915_debugfs_connector_add(struct drm_connector *connector)
{ return 0; }
static inline void intel_display_crc_init(struct drm_i915_private *dev_priv) {}
__printf(2, 3)
void i915_error_printf(struct drm_i915_error_state_buf *e, const char *f, ...);
int i915_error_state_to_str(struct drm_i915_error_state_buf *estr,
- const struct i915_error_state_file_priv *error);
+ const struct i915_gpu_state *gpu);
int i915_error_state_buf_init(struct drm_i915_error_state_buf *eb,
struct drm_i915_private *i915,
size_t count, loff_t pos);
{
kfree(eb->buf);
}
+
+struct i915_gpu_state *i915_capture_gpu_state(struct drm_i915_private *i915);
void i915_capture_error_state(struct drm_i915_private *dev_priv,
u32 engine_mask,
const char *error_msg);
-void i915_error_state_get(struct drm_device *dev,
- struct i915_error_state_file_priv *error_priv);
-void i915_error_state_put(struct i915_error_state_file_priv *error_priv);
-void i915_destroy_error_state(struct drm_i915_private *dev_priv);
+
+static inline struct i915_gpu_state *
+i915_gpu_state_get(struct i915_gpu_state *gpu)
+{
+ kref_get(&gpu->ref);
+ return gpu;
+}
+
+void __i915_gpu_state_free(struct kref *kref);
+static inline void i915_gpu_state_put(struct i915_gpu_state *gpu)
+{
+ if (gpu)
+ kref_put(&gpu->ref, __i915_gpu_state_free);
+}
+
+struct i915_gpu_state *i915_first_error_state(struct drm_i915_private *i915);
+void i915_reset_error_state(struct drm_i915_private *i915);
#else
{
}
-static inline void i915_destroy_error_state(struct drm_i915_private *dev_priv)
+static inline struct i915_gpu_state *
+i915_first_error_state(struct drm_i915_private *i915)
+{
+ return NULL;
+}
+
+static inline void i915_reset_error_state(struct drm_i915_private *i915)
{
}
extern void intel_i2c_reset(struct drm_i915_private *dev_priv);
/* intel_bios.c */
-int intel_bios_init(struct drm_i915_private *dev_priv);
+void intel_bios_init(struct drm_i915_private *dev_priv);
bool intel_bios_is_valid_vbt(const void *buf, size_t size);
bool intel_bios_is_tv_present(struct drm_i915_private *dev_priv);
bool intel_bios_is_lvds_present(struct drm_i915_private *dev_priv, u8 *i2c_pin);
extern void i915_redisable_vga_power_on(struct drm_i915_private *dev_priv);
extern bool ironlake_set_drps(struct drm_i915_private *dev_priv, u8 val);
extern void intel_init_pch_refclk(struct drm_i915_private *dev_priv);
-extern void intel_set_rps(struct drm_i915_private *dev_priv, u8 val);
+extern int intel_set_rps(struct drm_i915_private *dev_priv, u8 val);
extern bool intel_set_memory_cxsr(struct drm_i915_private *dev_priv,
bool enable);
extern struct intel_display_error_state *
intel_display_capture_error_state(struct drm_i915_private *dev_priv);
extern void intel_display_print_error_state(struct drm_i915_error_state_buf *e,
- struct drm_i915_private *dev_priv,
struct intel_display_error_state *error);
int sandybridge_pcode_read(struct drm_i915_private *dev_priv, u32 mbox, u32 *val);
/* intel_sideband.c */
u32 vlv_punit_read(struct drm_i915_private *dev_priv, u32 addr);
-void vlv_punit_write(struct drm_i915_private *dev_priv, u32 addr, u32 val);
+int vlv_punit_write(struct drm_i915_private *dev_priv, u32 addr, u32 val);
u32 vlv_nc_read(struct drm_i915_private *dev_priv, u8 addr);
u32 vlv_iosf_sb_read(struct drm_i915_private *dev_priv, u8 port, u32 reg);
void vlv_iosf_sb_write(struct drm_i915_private *dev_priv, u8 port, u32 reg, u32 val);
int intel_gpu_freq(struct drm_i915_private *dev_priv, int val);
int intel_freq_opcode(struct drm_i915_private *dev_priv, int val);
+u64 intel_rc6_residency_us(struct drm_i915_private *dev_priv,
+ const i915_reg_t reg);
#define I915_READ8(reg) dev_priv->uncore.funcs.mmio_readb(dev_priv, (reg), true)
#define I915_WRITE8(reg, val) dev_priv->uncore.funcs.mmio_writeb(dev_priv, (reg), (val), true)
}
static inline bool
-__i915_request_irq_complete(struct drm_i915_gem_request *req)
+__i915_request_irq_complete(const struct drm_i915_gem_request *req)
{
struct intel_engine_cs *engine = req->engine;
+ u32 seqno;
+
+ /* Note that the engine may have wrapped around the seqno, and
+ * so our request->global_seqno will be ahead of the hardware,
+ * even though it completed the request before wrapping. We catch
+ * this by kicking all the waiters before resetting the seqno
+ * in hardware, and also signal the fence.
+ */
+ if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &req->fence.flags))
+ return true;
+
+ /* The request was dequeued before we were awoken. We check after
+ * inspecting the hw to confirm that this was the same request
+ * that generated the HWS update. The memory barriers within
+ * the request execution are sufficient to ensure that a check
+ * after reading the value from hw matches this request.
+ */
+ seqno = i915_gem_request_global_seqno(req);
+ if (!seqno)
+ return false;
/* Before we do the heavier coherent read of the seqno,
* check the value (hopefully) in the CPU cacheline.
*/
- if (__i915_gem_request_completed(req))
+ if (__i915_gem_request_completed(req, seqno))
return true;
/* Ensure our read of the seqno is coherent so that we
* is woken.
*/
if (engine->irq_seqno_barrier &&
- rcu_access_pointer(engine->breadcrumbs.irq_seqno_bh) == current &&
- cmpxchg_relaxed(&engine->breadcrumbs.irq_posted, 1, 0)) {
- struct task_struct *tsk;
+ test_and_clear_bit(ENGINE_IRQ_BREADCRUMB, &engine->irq_posted)) {
+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
/* The ordering of irq_posted versus applying the barrier
* is crucial. The clearing of the current irq_posted must
* the seqno before we believe it coherent since they see
* irq_posted == false but we are still running).
*/
- rcu_read_lock();
- tsk = rcu_dereference(engine->breadcrumbs.irq_seqno_bh);
- if (tsk && tsk != current)
+ spin_lock_irq(&b->irq_lock);
+ if (b->irq_wait && b->irq_wait->tsk != current)
/* Note that if the bottom-half is changed as we
* are sending the wake-up, the new bottom-half will
* be woken by whomever made the change. We only have
* to worry about when we steal the irq-posted for
* ourself.
*/
- wake_up_process(tsk);
- rcu_read_unlock();
+ wake_up_process(b->irq_wait->tsk);
+ spin_unlock_irq(&b->irq_lock);
- if (__i915_gem_request_completed(req))
+ if (__i915_gem_request_completed(req, seqno))
return true;
}
unsigned long addr, unsigned long pfn, unsigned long size,
struct io_mapping *iomap);
+static inline bool i915_gem_object_is_coherent(struct drm_i915_gem_object *obj)
+{
+ return (obj->cache_level != I915_CACHE_NONE ||
+ HAS_LLC(to_i915(obj->base.dev)));
+}
+
#endif
#include <drm/drm_vma_manager.h>
#include <drm/i915_drm.h>
#include "i915_drv.h"
+#include "i915_gem_clflush.h"
#include "i915_vgpu.h"
#include "i915_trace.h"
#include "intel_drv.h"
#include "intel_frontbuffer.h"
#include "intel_mocs.h"
#include <linux/dma-fence-array.h>
+#include <linux/kthread.h>
#include <linux/reservation.h>
#include <linux/shmem_fs.h>
#include <linux/slab.h>
static void i915_gem_object_flush_gtt_write_domain(struct drm_i915_gem_object *obj);
static void i915_gem_object_flush_cpu_write_domain(struct drm_i915_gem_object *obj);
-static bool cpu_cache_is_coherent(struct drm_device *dev,
- enum i915_cache_level level)
-{
- return HAS_LLC(to_i915(dev)) || level != I915_CACHE_NONE;
-}
-
static bool cpu_write_needs_clflush(struct drm_i915_gem_object *obj)
{
if (obj->base.write_domain == I915_GEM_DOMAIN_CPU)
return false;
- if (!cpu_cache_is_coherent(obj->base.dev, obj->cache_level))
+ if (!i915_gem_object_is_coherent(obj))
return true;
return obj->pin_display;
might_sleep();
- if (!i915_reset_in_progress(error))
- return 0;
-
/*
* Only wait 10 seconds for the gpu reset to complete to avoid hanging
* userspace. If it takes that long something really bad is going on and
* we should simply try to bail out and fail as gracefully as possible.
*/
ret = wait_event_interruptible_timeout(error->reset_queue,
- !i915_reset_in_progress(error),
+ !i915_reset_backoff(error),
I915_RESET_TIMEOUT);
if (ret == 0) {
DRM_ERROR("Timed out waiting for the gpu reset to complete\n");
if (needs_clflush &&
(obj->base.read_domains & I915_GEM_DOMAIN_CPU) == 0 &&
- !cpu_cache_is_coherent(obj->base.dev, obj->cache_level))
+ !i915_gem_object_is_coherent(obj))
drm_clflush_sg(pages);
obj->base.read_domains = I915_GEM_DOMAIN_CPU;
.release = i915_gem_object_release_phys,
};
+static const struct drm_i915_gem_object_ops i915_gem_object_ops;
+
int i915_gem_object_unbind(struct drm_i915_gem_object *obj)
{
struct i915_vma *vma;
if (flags & I915_WAIT_LOCKED && i915_gem_request_completed(rq))
i915_gem_request_retire_upto(rq);
- if (rps && rq->global_seqno == intel_engine_last_submit(rq->engine)) {
+ if (rps && i915_gem_request_global_seqno(rq) == intel_engine_last_submit(rq->engine)) {
/* The GPU is now idle and this client has stalled.
* Since no other client has submitted a request in the
* meantime, assume that this client is the only one
long timeout,
struct intel_rps_client *rps)
{
+ unsigned int seq = __read_seqcount_begin(&resv->seq);
struct dma_fence *excl;
+ bool prune_fences = false;
if (flags & I915_WAIT_ALL) {
struct dma_fence **shared;
for (; i < count; i++)
dma_fence_put(shared[i]);
kfree(shared);
+
+ prune_fences = count && timeout >= 0;
} else {
excl = reservation_object_get_excl_rcu(resv);
}
- if (excl && timeout >= 0)
+ if (excl && timeout >= 0) {
timeout = i915_gem_object_wait_fence(excl, flags, timeout, rps);
+ prune_fences = timeout >= 0;
+ }
dma_fence_put(excl);
+ /* Oportunistically prune the fences iff we know they have *all* been
+ * signaled and that the reservation object has not been changed (i.e.
+ * no new fences have been added).
+ */
+ if (prune_fences && !__read_seqcount_retry(&resv->seq, seq)) {
+ if (reservation_object_trylock(resv)) {
+ if (!__read_seqcount_retry(&resv->seq, seq))
+ reservation_object_add_excl_fence(resv, NULL);
+ reservation_object_unlock(resv);
+ }
+ }
+
return timeout;
}
if (obj->mm.pages)
return -EBUSY;
+ GEM_BUG_ON(obj->ops != &i915_gem_object_ops);
obj->ops = &i915_gem_phys_ops;
- return i915_gem_object_pin_pages(obj);
+ ret = i915_gem_object_pin_pages(obj);
+ if (ret)
+ goto err_xfer;
+
+ return 0;
+
+err_xfer:
+ obj->ops = &i915_gem_object_ops;
+ return ret;
}
static int
drm_clflush_virt_range(vaddr, args->size);
i915_gem_chipset_flush(to_i915(obj->base.dev));
- intel_fb_obj_flush(obj, false, ORIGIN_CPU);
+ intel_fb_obj_flush(obj, ORIGIN_CPU);
return 0;
}
if (ret)
return ret;
+ if (i915_gem_object_is_coherent(obj) ||
+ !static_cpu_has(X86_FEATURE_CLFLUSH)) {
+ ret = i915_gem_object_set_to_cpu_domain(obj, false);
+ if (ret)
+ goto err_unpin;
+ else
+ goto out;
+ }
+
i915_gem_object_flush_gtt_write_domain(obj);
/* If we're not in the cpu read domain, set ourself into the gtt
* anyway again before the next pread happens.
*/
if (!(obj->base.read_domains & I915_GEM_DOMAIN_CPU))
- *needs_clflush = !cpu_cache_is_coherent(obj->base.dev,
- obj->cache_level);
-
- if (*needs_clflush && !static_cpu_has(X86_FEATURE_CLFLUSH)) {
- ret = i915_gem_object_set_to_cpu_domain(obj, false);
- if (ret)
- goto err_unpin;
-
- *needs_clflush = 0;
- }
+ *needs_clflush = CLFLUSH_BEFORE;
+out:
/* return with the pages pinned */
return 0;
if (ret)
return ret;
+ if (i915_gem_object_is_coherent(obj) ||
+ !static_cpu_has(X86_FEATURE_CLFLUSH)) {
+ ret = i915_gem_object_set_to_cpu_domain(obj, true);
+ if (ret)
+ goto err_unpin;
+ else
+ goto out;
+ }
+
i915_gem_object_flush_gtt_write_domain(obj);
/* If we're not in the cpu write domain, set ourself into the
* right away and we therefore have to clflush anyway.
*/
if (obj->base.write_domain != I915_GEM_DOMAIN_CPU)
- *needs_clflush |= cpu_write_needs_clflush(obj) << 1;
+ *needs_clflush |= CLFLUSH_AFTER;
/* Same trick applies to invalidate partially written cachelines read
* before writing.
*/
if (!(obj->base.read_domains & I915_GEM_DOMAIN_CPU))
- *needs_clflush |= !cpu_cache_is_coherent(obj->base.dev,
- obj->cache_level);
-
- if (*needs_clflush && !static_cpu_has(X86_FEATURE_CLFLUSH)) {
- ret = i915_gem_object_set_to_cpu_domain(obj, true);
- if (ret)
- goto err_unpin;
-
- *needs_clflush = 0;
- }
-
- if ((*needs_clflush & CLFLUSH_AFTER) == 0)
- obj->cache_dirty = true;
+ *needs_clflush |= CLFLUSH_BEFORE;
+out:
intel_fb_obj_invalidate(obj, ORIGIN_CPU);
obj->mm.dirty = true;
/* return with the pages pinned */
user_data += page_length;
offset += page_length;
}
- intel_fb_obj_flush(obj, false, ORIGIN_CPU);
+ intel_fb_obj_flush(obj, ORIGIN_CPU);
mutex_lock(&i915->drm.struct_mutex);
out_unpin:
offset = 0;
}
- intel_fb_obj_flush(obj, false, ORIGIN_CPU);
+ intel_fb_obj_flush(obj, ORIGIN_CPU);
i915_gem_obj_finish_shmem_access(obj);
return ret;
}
{
struct drm_i915_gem_sw_finish *args = data;
struct drm_i915_gem_object *obj;
- int err = 0;
obj = i915_gem_object_lookup(file, args->handle);
if (!obj)
return -ENOENT;
/* Pinned buffers may be scanout, so flush the cache */
- if (READ_ONCE(obj->pin_display)) {
- err = i915_mutex_lock_interruptible(dev);
- if (!err) {
- i915_gem_object_flush_cpu_write_domain(obj);
- mutex_unlock(&dev->struct_mutex);
- }
- }
-
+ i915_gem_object_flush_if_display(obj);
i915_gem_object_put(obj);
- return err;
+
+ return 0;
}
/**
mutex_unlock(&obj->mm.lock);
}
-static void i915_sg_trim(struct sg_table *orig_st)
+static bool i915_sg_trim(struct sg_table *orig_st)
{
struct sg_table new_st;
struct scatterlist *sg, *new_sg;
unsigned int i;
if (orig_st->nents == orig_st->orig_nents)
- return;
+ return false;
if (sg_alloc_table(&new_st, orig_st->nents, GFP_KERNEL | __GFP_NOWARN))
- return;
+ return false;
new_sg = new_st.sgl;
for_each_sg(orig_st->sgl, sg, orig_st->nents, i) {
sg_free_table(orig_st);
*orig_st = new_st;
+ return true;
}
static struct sg_table *
struct drm_i915_gem_request *
i915_gem_find_active_request(struct intel_engine_cs *engine)
{
- struct drm_i915_gem_request *request;
+ struct drm_i915_gem_request *request, *active = NULL;
+ unsigned long flags;
/* We are called by the error capture and reset at a random
* point in time. In particular, note that neither is crucially
* extra delay for a recent interrupt is pointless. Hence, we do
* not need an engine->irq_seqno_barrier() before the seqno reads.
*/
+ spin_lock_irqsave(&engine->timeline->lock, flags);
list_for_each_entry(request, &engine->timeline->requests, link) {
- if (__i915_gem_request_completed(request))
+ if (__i915_gem_request_completed(request,
+ request->global_seqno))
continue;
GEM_BUG_ON(request->engine != engine);
- return request;
+ GEM_BUG_ON(test_bit(DMA_FENCE_FLAG_SIGNALED_BIT,
+ &request->fence.flags));
+
+ active = request;
+ break;
}
+ spin_unlock_irqrestore(&engine->timeline->lock, flags);
- return NULL;
+ return active;
}
static bool engine_stalled(struct intel_engine_cs *engine)
for_each_engine(engine, dev_priv, id) {
struct drm_i915_gem_request *request;
+ /* Prevent the signaler thread from updating the request
+ * state (by calling dma_fence_signal) as we are processing
+ * the reset. The write from the GPU of the seqno is
+ * asynchronous and the signaler thread may see a different
+ * value to us and declare the request complete, even though
+ * the reset routine have picked that request as the active
+ * (incomplete) request. This conflict is not handled
+ * gracefully!
+ */
+ kthread_park(engine->breadcrumbs.signaler);
+
/* Prevent request submission to the hardware until we have
* completed the reset in i915_gem_reset_finish(). If a request
* is completed by one engine, it may then queue a request
tasklet_kill(&engine->irq_tasklet);
tasklet_disable(&engine->irq_tasklet);
+ if (engine->irq_seqno_barrier)
+ engine->irq_seqno_barrier(engine);
+
if (engine_stalled(engine)) {
request = i915_gem_find_active_request(engine);
if (request && request->fence.error == -EIO)
{
struct drm_i915_gem_request *request;
- if (engine->irq_seqno_barrier)
- engine->irq_seqno_barrier(engine);
-
request = i915_gem_find_active_request(engine);
if (request && i915_gem_reset_request(request)) {
DRM_DEBUG_DRIVER("resetting %s to restart from tail of request 0x%x\n",
i915_gem_retire_requests(dev_priv);
- for_each_engine(engine, dev_priv, id)
+ for_each_engine(engine, dev_priv, id) {
+ struct i915_gem_context *ctx;
+
i915_gem_reset_engine(engine);
+ ctx = fetch_and_zero(&engine->last_retired_context);
+ if (ctx)
+ engine->context_unpin(engine, ctx);
+ }
i915_gem_restore_fences(dev_priv);
lockdep_assert_held(&dev_priv->drm.struct_mutex);
- for_each_engine(engine, dev_priv, id)
+ for_each_engine(engine, dev_priv, id) {
tasklet_enable(&engine->irq_tasklet);
+ kthread_unpark(engine->breadcrumbs.signaler);
+ }
}
static void nop_submit_request(struct drm_i915_gem_request *request)
mod_delayed_work(dev_priv->wq, &dev_priv->gt.idle_work, 0);
}
+bool i915_gem_unset_wedged(struct drm_i915_private *i915)
+{
+ struct i915_gem_timeline *tl;
+ int i;
+
+ lockdep_assert_held(&i915->drm.struct_mutex);
+ if (!test_bit(I915_WEDGED, &i915->gpu_error.flags))
+ return true;
+
+ /* Before unwedging, make sure that all pending operations
+ * are flushed and errored out - we may have requests waiting upon
+ * third party fences. We marked all inflight requests as EIO, and
+ * every execbuf since returned EIO, for consistency we want all
+ * the currently pending requests to also be marked as EIO, which
+ * is done inside our nop_submit_request - and so we must wait.
+ *
+ * No more can be submitted until we reset the wedged bit.
+ */
+ list_for_each_entry(tl, &i915->gt.timelines, link) {
+ for (i = 0; i < ARRAY_SIZE(tl->engine); i++) {
+ struct drm_i915_gem_request *rq;
+
+ rq = i915_gem_active_peek(&tl->engine[i].last_request,
+ &i915->drm.struct_mutex);
+ if (!rq)
+ continue;
+
+ /* We can't use our normal waiter as we want to
+ * avoid recursively trying to handle the current
+ * reset. The basic dma_fence_default_wait() installs
+ * a callback for dma_fence_signal(), which is
+ * triggered by our nop handler (indirectly, the
+ * callback enables the signaler thread which is
+ * woken by the nop_submit_request() advancing the seqno
+ * and when the seqno passes the fence, the signaler
+ * then signals the fence waking us up).
+ */
+ if (dma_fence_default_wait(&rq->fence, true,
+ MAX_SCHEDULE_TIMEOUT) < 0)
+ return false;
+ }
+ }
+
+ /* Undo nop_submit_request. We prevent all new i915 requests from
+ * being queued (by disallowing execbuf whilst wedged) so having
+ * waited for all active requests above, we know the system is idle
+ * and do not have to worry about a thread being inside
+ * engine->submit_request() as we swap over. So unlike installing
+ * the nop_submit_request on reset, we can do this from normal
+ * context and do not require stop_machine().
+ */
+ intel_engines_reset_default_submission(i915);
+
+ smp_mb__before_atomic(); /* complete takeover before enabling execbuf */
+ clear_bit(I915_WEDGED, &i915->gpu_error.flags);
+
+ return true;
+}
+
static void
i915_gem_retire_work_handler(struct work_struct *work)
{
* new request is submitted.
*/
wait_for(READ_ONCE(dev_priv->gt.active_requests) ||
- intel_execlists_idle(dev_priv), 10);
-
+ intel_engines_are_idle(dev_priv),
+ 10);
if (READ_ONCE(dev_priv->gt.active_requests))
return;
if (dev_priv->gt.active_requests)
goto out_unlock;
- if (wait_for(intel_execlists_idle(dev_priv), 10))
+ if (wait_for(intel_engines_are_idle(dev_priv), 10))
DRM_ERROR("Timeout waiting for engines to idle\n");
- for_each_engine(engine, dev_priv, id)
+ for_each_engine(engine, dev_priv, id) {
+ intel_engine_disarm_breadcrumbs(engine);
i915_gem_batch_pool_fini(&engine->batch_pool);
+ }
GEM_BUG_ON(!dev_priv->gt.awake);
dev_priv->gt.awake = false;
return 0;
}
-void i915_gem_clflush_object(struct drm_i915_gem_object *obj,
- bool force)
-{
- /* If we don't have a page list set up, then we're not pinned
- * to GPU, and we can ignore the cache flush because it'll happen
- * again at bind time.
- */
- if (!obj->mm.pages)
- return;
-
- /*
- * Stolen memory is always coherent with the GPU as it is explicitly
- * marked as wc by the system, or the system is cache-coherent.
- */
- if (obj->stolen || obj->phys_handle)
- return;
-
- /* If the GPU is snooping the contents of the CPU cache,
- * we do not need to manually clear the CPU cache lines. However,
- * the caches are only snooped when the render cache is
- * flushed/invalidated. As we always have to emit invalidations
- * and flushes when moving into and out of the RENDER domain, correct
- * snooping behaviour occurs naturally as the result of our domain
- * tracking.
- */
- if (!force && cpu_cache_is_coherent(obj->base.dev, obj->cache_level)) {
- obj->cache_dirty = true;
- return;
- }
-
- trace_i915_gem_object_clflush(obj);
- drm_clflush_sg(obj->mm.pages);
- obj->cache_dirty = false;
-}
-
/** Flushes the GTT write domain for the object if it's dirty. */
static void
i915_gem_object_flush_gtt_write_domain(struct drm_i915_gem_object *obj)
if (INTEL_GEN(dev_priv) >= 6 && !HAS_LLC(dev_priv))
POSTING_READ(RING_ACTHD(dev_priv->engine[RCS]->mmio_base));
- intel_fb_obj_flush(obj, false, write_origin(obj, I915_GEM_DOMAIN_GTT));
+ intel_fb_obj_flush(obj, write_origin(obj, I915_GEM_DOMAIN_GTT));
obj->base.write_domain = 0;
- trace_i915_gem_object_change_domain(obj,
- obj->base.read_domains,
- I915_GEM_DOMAIN_GTT);
}
/** Flushes the CPU write domain for the object if it's dirty. */
if (obj->base.write_domain != I915_GEM_DOMAIN_CPU)
return;
- i915_gem_clflush_object(obj, obj->pin_display);
- intel_fb_obj_flush(obj, false, ORIGIN_CPU);
+ i915_gem_clflush_object(obj, I915_CLFLUSH_SYNC);
+ obj->base.write_domain = 0;
+}
+static void __i915_gem_object_flush_for_display(struct drm_i915_gem_object *obj)
+{
+ if (obj->base.write_domain != I915_GEM_DOMAIN_CPU && !obj->cache_dirty)
+ return;
+
+ i915_gem_clflush_object(obj, I915_CLFLUSH_FORCE);
obj->base.write_domain = 0;
- trace_i915_gem_object_change_domain(obj,
- obj->base.read_domains,
- I915_GEM_DOMAIN_CPU);
+}
+
+void i915_gem_object_flush_if_display(struct drm_i915_gem_object *obj)
+{
+ if (!READ_ONCE(obj->pin_display))
+ return;
+
+ mutex_lock(&obj->base.dev->struct_mutex);
+ __i915_gem_object_flush_for_display(obj);
+ mutex_unlock(&obj->base.dev->struct_mutex);
}
/**
int
i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write)
{
- uint32_t old_write_domain, old_read_domains;
int ret;
lockdep_assert_held(&obj->base.dev->struct_mutex);
if ((obj->base.read_domains & I915_GEM_DOMAIN_GTT) == 0)
mb();
- old_write_domain = obj->base.write_domain;
- old_read_domains = obj->base.read_domains;
-
/* It should now be out of any other write domains, and we can update
* the domain values for our changes.
*/
obj->mm.dirty = true;
}
- trace_i915_gem_object_change_domain(obj,
- old_read_domains,
- old_write_domain);
-
i915_gem_object_unpin_pages(obj);
return 0;
}
}
if (obj->base.write_domain == I915_GEM_DOMAIN_CPU &&
- cpu_cache_is_coherent(obj->base.dev, obj->cache_level))
+ i915_gem_object_is_coherent(obj))
obj->cache_dirty = true;
list_for_each_entry(vma, &obj->vma_list, obj_link)
const struct i915_ggtt_view *view)
{
struct i915_vma *vma;
- u32 old_read_domains, old_write_domain;
int ret;
lockdep_assert_held(&obj->base.dev->struct_mutex);
vma->display_alignment = max_t(u64, vma->display_alignment, alignment);
/* Treat this as an end-of-frame, like intel_user_framebuffer_dirty() */
- if (obj->cache_dirty || obj->base.write_domain == I915_GEM_DOMAIN_CPU) {
- i915_gem_clflush_object(obj, true);
- intel_fb_obj_flush(obj, false, ORIGIN_DIRTYFB);
- }
-
- old_write_domain = obj->base.write_domain;
- old_read_domains = obj->base.read_domains;
+ __i915_gem_object_flush_for_display(obj);
+ intel_fb_obj_flush(obj, ORIGIN_DIRTYFB);
/* It should now be out of any other write domains, and we can update
* the domain values for our changes.
*/
- obj->base.write_domain = 0;
obj->base.read_domains |= I915_GEM_DOMAIN_GTT;
- trace_i915_gem_object_change_domain(obj,
- old_read_domains,
- old_write_domain);
-
return vma;
err_unpin_display:
int
i915_gem_object_set_to_cpu_domain(struct drm_i915_gem_object *obj, bool write)
{
- uint32_t old_write_domain, old_read_domains;
int ret;
lockdep_assert_held(&obj->base.dev->struct_mutex);
i915_gem_object_flush_gtt_write_domain(obj);
- old_write_domain = obj->base.write_domain;
- old_read_domains = obj->base.read_domains;
-
/* Flush the CPU cache if it's still invalid. */
if ((obj->base.read_domains & I915_GEM_DOMAIN_CPU) == 0) {
- i915_gem_clflush_object(obj, false);
-
+ i915_gem_clflush_object(obj, I915_CLFLUSH_SYNC);
obj->base.read_domains |= I915_GEM_DOMAIN_CPU;
}
obj->base.write_domain = I915_GEM_DOMAIN_CPU;
}
- trace_i915_gem_object_change_domain(obj,
- old_read_domains,
- old_write_domain);
-
return 0;
}
return -EIO;
spin_lock(&file_priv->mm.lock);
- list_for_each_entry(request, &file_priv->mm.request_list, client_list) {
+ list_for_each_entry(request, &file_priv->mm.request_list, client_link) {
if (time_after_eq(request->emitted_jiffies, recent_enough))
break;
- /*
- * Note that the request might not have been submitted yet.
- * In which case emitted_jiffies will be zero.
- */
- if (!request->emitted_jiffies)
- continue;
+ if (target) {
+ list_del(&target->client_link);
+ target->file_priv = NULL;
+ }
target = request;
}
struct drm_i915_gem_object *obj =
container_of(active, typeof(*obj), frontbuffer_write);
- intel_fb_obj_flush(obj, true, ORIGIN_CS);
+ intel_fb_obj_flush(obj, ORIGIN_CS);
}
void i915_gem_object_init(struct drm_i915_gem_object *obj,
!i915_gem_context_is_kernel(engine->last_retired_context));
}
+void i915_gem_sanitize(struct drm_i915_private *i915)
+{
+ /*
+ * If we inherit context state from the BIOS or earlier occupants
+ * of the GPU, the GPU may be in an inconsistent state when we
+ * try to take over. The only way to remove the earlier state
+ * is by resetting. However, resetting on earlier gen is tricky as
+ * it may impact the display and we are uncertain about the stability
+ * of the reset, so we only reset recent machines with logical
+ * context support (that must be reset to remove any stray contexts).
+ */
+ if (HAS_HW_CONTEXTS(i915)) {
+ int reset = intel_gpu_reset(i915, ALL_ENGINES);
+ WARN_ON(reset && reset != -ENODEV);
+ }
+}
+
int i915_gem_suspend(struct drm_i915_private *dev_priv)
{
struct drm_device *dev = &dev_priv->drm;
int ret;
+ intel_runtime_pm_get(dev_priv);
intel_suspend_gt_powersave(dev_priv);
mutex_lock(&dev->struct_mutex);
*/
ret = i915_gem_switch_to_kernel_context(dev_priv);
if (ret)
- goto err;
+ goto err_unlock;
ret = i915_gem_wait_for_idle(dev_priv,
I915_WAIT_INTERRUPTIBLE |
I915_WAIT_LOCKED);
if (ret)
- goto err;
+ goto err_unlock;
i915_gem_retire_requests(dev_priv);
GEM_BUG_ON(dev_priv->gt.active_requests);
* reset the GPU back to its idle, low power state.
*/
WARN_ON(dev_priv->gt.awake);
- WARN_ON(!intel_execlists_idle(dev_priv));
+ WARN_ON(!intel_engines_are_idle(dev_priv));
/*
* Neither the BIOS, ourselves or any other kernel
* machines is a good idea, we don't - just in case it leaves the
* machine in an unusable condition.
*/
- if (HAS_HW_CONTEXTS(dev_priv)) {
- int reset = intel_gpu_reset(dev_priv, ALL_ENGINES);
- WARN_ON(reset && reset != -ENODEV);
- }
+ i915_gem_sanitize(dev_priv);
+ goto out_rpm_put;
- return 0;
-
-err:
+err_unlock:
mutex_unlock(&dev->struct_mutex);
+out_rpm_put:
+ intel_runtime_pm_put(dev_priv);
return ret;
}
}
}
-int
-i915_gem_init_hw(struct drm_i915_private *dev_priv)
+static int __i915_gem_restart_engines(void *data)
{
+ struct drm_i915_private *i915 = data;
struct intel_engine_cs *engine;
enum intel_engine_id id;
+ int err;
+
+ for_each_engine(engine, i915, id) {
+ err = engine->init_hw(engine);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+int i915_gem_init_hw(struct drm_i915_private *dev_priv)
+{
int ret;
dev_priv->gt.last_init_time = ktime_get();
}
/* Need to do basic initialisation of all rings first: */
- for_each_engine(engine, dev_priv, id) {
- ret = engine->init_hw(engine);
- if (ret)
- goto out;
- }
+ ret = __i915_gem_restart_engines(dev_priv);
+ if (ret)
+ goto out;
intel_mocs_init_l3cc_table(dev_priv);
/* We can't enable contexts until all firmware is loaded */
- ret = intel_guc_setup(dev_priv);
+ ret = intel_uc_init_hw(dev_priv);
if (ret)
goto out;
mutex_lock(&dev_priv->drm.struct_mutex);
+ i915_gem_clflush_init(dev_priv);
+
if (!i915.enable_execlists) {
dev_priv->gt.resume = intel_legacy_submission_resume;
dev_priv->gt.cleanup_engine = intel_engine_cleanup;
return ret;
}
+void i915_gem_init_mmio(struct drm_i915_private *i915)
+{
+ i915_gem_sanitize(i915);
+}
+
void
i915_gem_cleanup_engines(struct drm_i915_private *dev_priv)
{
void i915_gem_load_cleanup(struct drm_i915_private *dev_priv)
{
+ i915_gem_drain_freed_objects(dev_priv);
WARN_ON(!llist_empty(&dev_priv->mm.free_list));
+ WARN_ON(dev_priv->mm.object_count);
mutex_lock(&dev_priv->drm.struct_mutex);
i915_gem_timeline_fini(&dev_priv->gt.global_timeline);
int i915_gem_freeze(struct drm_i915_private *dev_priv)
{
- intel_runtime_pm_get(dev_priv);
-
mutex_lock(&dev_priv->drm.struct_mutex);
i915_gem_shrink_all(dev_priv);
mutex_unlock(&dev_priv->drm.struct_mutex);
- intel_runtime_pm_put(dev_priv);
-
return 0;
}
* file_priv.
*/
spin_lock(&file_priv->mm.lock);
- list_for_each_entry(request, &file_priv->mm.request_list, client_list)
+ list_for_each_entry(request, &file_priv->mm.request_list, client_link)
request->file_priv = NULL;
spin_unlock(&file_priv->mm.lock);
const void *data, size_t size)
{
struct drm_i915_gem_object *obj;
- struct sg_table *sg;
- size_t bytes;
- int ret;
+ struct file *file;
+ size_t offset;
+ int err;
obj = i915_gem_object_create(dev_priv, round_up(size, PAGE_SIZE));
if (IS_ERR(obj))
return obj;
- ret = i915_gem_object_set_to_cpu_domain(obj, true);
- if (ret)
- goto fail;
+ GEM_BUG_ON(obj->base.write_domain != I915_GEM_DOMAIN_CPU);
- ret = i915_gem_object_pin_pages(obj);
- if (ret)
- goto fail;
+ file = obj->base.filp;
+ offset = 0;
+ do {
+ unsigned int len = min_t(typeof(size), size, PAGE_SIZE);
+ struct page *page;
+ void *pgdata, *vaddr;
- sg = obj->mm.pages;
- bytes = sg_copy_from_buffer(sg->sgl, sg->nents, (void *)data, size);
- obj->mm.dirty = true; /* Backing store is now out of date */
- i915_gem_object_unpin_pages(obj);
+ err = pagecache_write_begin(file, file->f_mapping,
+ offset, len, 0,
+ &page, &pgdata);
+ if (err < 0)
+ goto fail;
- if (WARN_ON(bytes != size)) {
- DRM_ERROR("Incomplete copy, wrote %zu of %zu", bytes, size);
- ret = -EFAULT;
- goto fail;
- }
+ vaddr = kmap(page);
+ memcpy(vaddr, data, len);
+ kunmap(page);
+
+ err = pagecache_write_end(file, file->f_mapping,
+ offset, len, len,
+ page, pgdata);
+ if (err < 0)
+ goto fail;
+
+ size -= len;
+ data += len;
+ offset += len;
+ } while (size);
return obj;
fail:
i915_gem_object_put(obj);
- return ERR_PTR(ret);
+ return ERR_PTR(err);
}
struct scatterlist *
sg = i915_gem_object_get_sg(obj, n, &offset);
return sg_dma_address(sg) + (offset << PAGE_SHIFT);
}
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/scatterlist.c"
+#include "selftests/mock_gem_device.c"
+#include "selftests/huge_gem_object.c"
+#include "selftests/i915_gem_object.c"
+#include "selftests/i915_gem_coherency.c"
+#endif
#ifdef CONFIG_DRM_I915_DEBUG_GEM
#define GEM_BUG_ON(expr) BUG_ON(expr)
#define GEM_WARN_ON(expr) WARN_ON(expr)
+
+#define GEM_DEBUG_DECL(var) var
+#define GEM_DEBUG_EXEC(expr) expr
+#define GEM_DEBUG_BUG_ON(expr) GEM_BUG_ON(expr)
+
#else
#define GEM_BUG_ON(expr) BUILD_BUG_ON_INVALID(expr)
#define GEM_WARN_ON(expr) (BUILD_BUG_ON_INVALID(expr), 0)
+
+#define GEM_DEBUG_DECL(var)
+#define GEM_DEBUG_EXEC(expr) do { } while (0)
+#define GEM_DEBUG_BUG_ON(expr)
#endif
#define I915_NUM_ENGINES 5
i915_gem_batch_pool_get(struct i915_gem_batch_pool *pool,
size_t size)
{
- struct drm_i915_gem_object *obj = NULL;
- struct drm_i915_gem_object *tmp;
+ struct drm_i915_gem_object *obj;
struct list_head *list;
int n, ret;
n = ARRAY_SIZE(pool->cache_list) - 1;
list = &pool->cache_list[n];
- list_for_each_entry(tmp, list, batch_pool_link) {
+ list_for_each_entry(obj, list, batch_pool_link) {
/* The batches are strictly LRU ordered */
- if (i915_gem_object_is_active(tmp))
- break;
+ if (i915_gem_object_is_active(obj)) {
+ if (!reservation_object_test_signaled_rcu(obj->resv,
+ true))
+ break;
- GEM_BUG_ON(!reservation_object_test_signaled_rcu(tmp->resv,
- true));
+ i915_gem_retire_requests(pool->engine->i915);
+ GEM_BUG_ON(i915_gem_object_is_active(obj));
+ }
- if (tmp->base.size >= size) {
- /* Clear the set of shared fences early */
- ww_mutex_lock(&tmp->resv->lock, NULL);
- reservation_object_add_excl_fence(tmp->resv, NULL);
- ww_mutex_unlock(&tmp->resv->lock);
+ GEM_BUG_ON(!reservation_object_test_signaled_rcu(obj->resv,
+ true));
- obj = tmp;
- break;
- }
+ if (obj->base.size >= size)
+ goto found;
}
- if (obj == NULL) {
- obj = i915_gem_object_create_internal(pool->engine->i915, size);
- if (IS_ERR(obj))
- return obj;
- }
+ obj = i915_gem_object_create_internal(pool->engine->i915, size);
+ if (IS_ERR(obj))
+ return obj;
+found:
ret = i915_gem_object_pin_pages(obj);
if (ret)
return ERR_PTR(ret);
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "i915_drv.h"
+#include "intel_frontbuffer.h"
+#include "i915_gem_clflush.h"
+
+static DEFINE_SPINLOCK(clflush_lock);
+static u64 clflush_context;
+
+struct clflush {
+ struct dma_fence dma; /* Must be first for dma_fence_free() */
+ struct i915_sw_fence wait;
+ struct work_struct work;
+ struct drm_i915_gem_object *obj;
+};
+
+static const char *i915_clflush_get_driver_name(struct dma_fence *fence)
+{
+ return DRIVER_NAME;
+}
+
+static const char *i915_clflush_get_timeline_name(struct dma_fence *fence)
+{
+ return "clflush";
+}
+
+static bool i915_clflush_enable_signaling(struct dma_fence *fence)
+{
+ return true;
+}
+
+static void i915_clflush_release(struct dma_fence *fence)
+{
+ struct clflush *clflush = container_of(fence, typeof(*clflush), dma);
+
+ i915_sw_fence_fini(&clflush->wait);
+
+ BUILD_BUG_ON(offsetof(typeof(*clflush), dma));
+ dma_fence_free(&clflush->dma);
+}
+
+static const struct dma_fence_ops i915_clflush_ops = {
+ .get_driver_name = i915_clflush_get_driver_name,
+ .get_timeline_name = i915_clflush_get_timeline_name,
+ .enable_signaling = i915_clflush_enable_signaling,
+ .wait = dma_fence_default_wait,
+ .release = i915_clflush_release,
+};
+
+static void __i915_do_clflush(struct drm_i915_gem_object *obj)
+{
+ drm_clflush_sg(obj->mm.pages);
+ obj->cache_dirty = false;
+
+ intel_fb_obj_flush(obj, ORIGIN_CPU);
+}
+
+static void i915_clflush_work(struct work_struct *work)
+{
+ struct clflush *clflush = container_of(work, typeof(*clflush), work);
+ struct drm_i915_gem_object *obj = clflush->obj;
+
+ if (!obj->cache_dirty)
+ goto out;
+
+ if (i915_gem_object_pin_pages(obj)) {
+ DRM_ERROR("Failed to acquire obj->pages for clflushing\n");
+ goto out;
+ }
+
+ __i915_do_clflush(obj);
+
+ i915_gem_object_unpin_pages(obj);
+
+out:
+ i915_gem_object_put(obj);
+
+ dma_fence_signal(&clflush->dma);
+ dma_fence_put(&clflush->dma);
+}
+
+static int __i915_sw_fence_call
+i915_clflush_notify(struct i915_sw_fence *fence,
+ enum i915_sw_fence_notify state)
+{
+ struct clflush *clflush = container_of(fence, typeof(*clflush), wait);
+
+ switch (state) {
+ case FENCE_COMPLETE:
+ schedule_work(&clflush->work);
+ break;
+
+ case FENCE_FREE:
+ dma_fence_put(&clflush->dma);
+ break;
+ }
+
+ return NOTIFY_DONE;
+}
+
+void i915_gem_clflush_object(struct drm_i915_gem_object *obj,
+ unsigned int flags)
+{
+ struct clflush *clflush;
+
+ /*
+ * Stolen memory is always coherent with the GPU as it is explicitly
+ * marked as wc by the system, or the system is cache-coherent.
+ * Similarly, we only access struct pages through the CPU cache, so
+ * anything not backed by physical memory we consider to be always
+ * coherent and not need clflushing.
+ */
+ if (!i915_gem_object_has_struct_page(obj))
+ return;
+
+ obj->cache_dirty = true;
+
+ /* If the GPU is snooping the contents of the CPU cache,
+ * we do not need to manually clear the CPU cache lines. However,
+ * the caches are only snooped when the render cache is
+ * flushed/invalidated. As we always have to emit invalidations
+ * and flushes when moving into and out of the RENDER domain, correct
+ * snooping behaviour occurs naturally as the result of our domain
+ * tracking.
+ */
+ if (!(flags & I915_CLFLUSH_FORCE) && i915_gem_object_is_coherent(obj))
+ return;
+
+ trace_i915_gem_object_clflush(obj);
+
+ clflush = NULL;
+ if (!(flags & I915_CLFLUSH_SYNC))
+ clflush = kmalloc(sizeof(*clflush), GFP_KERNEL);
+ if (clflush) {
+ dma_fence_init(&clflush->dma,
+ &i915_clflush_ops,
+ &clflush_lock,
+ clflush_context,
+ 0);
+ i915_sw_fence_init(&clflush->wait, i915_clflush_notify);
+
+ clflush->obj = i915_gem_object_get(obj);
+ INIT_WORK(&clflush->work, i915_clflush_work);
+
+ dma_fence_get(&clflush->dma);
+
+ i915_sw_fence_await_reservation(&clflush->wait,
+ obj->resv, NULL,
+ false, I915_FENCE_TIMEOUT,
+ GFP_KERNEL);
+
+ reservation_object_lock(obj->resv, NULL);
+ reservation_object_add_excl_fence(obj->resv, &clflush->dma);
+ reservation_object_unlock(obj->resv);
+
+ i915_sw_fence_commit(&clflush->wait);
+ } else if (obj->mm.pages) {
+ __i915_do_clflush(obj);
+ } else {
+ GEM_BUG_ON(obj->base.write_domain != I915_GEM_DOMAIN_CPU);
+ }
+}
+
+void i915_gem_clflush_init(struct drm_i915_private *i915)
+{
+ clflush_context = dma_fence_context_alloc(1);
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __I915_GEM_CLFLUSH_H__
+#define __I915_GEM_CLFLUSH_H__
+
+struct drm_i915_private;
+struct drm_i915_gem_object;
+
+void i915_gem_clflush_init(struct drm_i915_private *i915);
+void i915_gem_clflush_object(struct drm_i915_gem_object *obj,
+ unsigned int flags);
+#define I915_CLFLUSH_FORCE BIT(0)
+#define I915_CLFLUSH_SYNC BIT(1)
+
+#endif /* __I915_GEM_CLFLUSH_H__ */
#define ALL_L3_SLICES(dev) (1 << NUM_L3_SLICES(dev)) - 1
-/* This is a HW constraint. The value below is the largest known requirement
- * I've seen in a spec to date, and that was a workaround for a non-shipping
- * part. It should be safe to decrease this, but it's more future proof as is.
- */
-#define GEN6_CONTEXT_ALIGN (64<<10)
-#define GEN7_CONTEXT_ALIGN I915_GTT_MIN_ALIGNMENT
-
-static size_t get_context_alignment(struct drm_i915_private *dev_priv)
-{
- if (IS_GEN6(dev_priv))
- return GEN6_CONTEXT_ALIGN;
-
- return GEN7_CONTEXT_ALIGN;
-}
-
static int get_context_size(struct drm_i915_private *dev_priv)
{
int ret;
return 0;
}
+static u32 default_desc_template(const struct drm_i915_private *i915,
+ const struct i915_hw_ppgtt *ppgtt)
+{
+ u32 address_mode;
+ u32 desc;
+
+ desc = GEN8_CTX_VALID | GEN8_CTX_PRIVILEGE;
+
+ address_mode = INTEL_LEGACY_32B_CONTEXT;
+ if (ppgtt && i915_vm_is_48bit(&ppgtt->base))
+ address_mode = INTEL_LEGACY_64B_CONTEXT;
+ desc |= address_mode << GEN8_CTX_ADDRESSING_MODE_SHIFT;
+
+ if (IS_GEN8(i915))
+ desc |= GEN8_CTX_L3LLC_COHERENT;
+
+ /* TODO: WaDisableLiteRestore when we start using semaphore
+ * signalling between Command Streamers
+ * ring->ctx_desc_template |= GEN8_CTX_FORCE_RESTORE;
+ */
+
+ return desc;
+}
+
static struct i915_gem_context *
__create_hw_context(struct drm_i915_private *dev_priv,
struct drm_i915_file_private *file_priv)
list_add_tail(&ctx->link, &dev_priv->context_list);
ctx->i915 = dev_priv;
- ctx->ggtt_alignment = get_context_alignment(dev_priv);
-
if (dev_priv->hw_context_size) {
struct drm_i915_gem_object *obj;
struct i915_vma *vma;
i915_gem_context_set_bannable(ctx);
ctx->ring_size = 4 * PAGE_SIZE;
- ctx->desc_template = GEN8_CTX_ADDRESSING_MODE(dev_priv) <<
- GEN8_CTX_ADDRESSING_MODE_SHIFT;
+ ctx->desc_template =
+ default_desc_template(dev_priv, dev_priv->mm.aliasing_ppgtt);
/* GuC requires the ring to be placed above GUC_WOPCM_TOP. If GuC is not
* present or not in use we still need a small bias as ring wraparound
return ERR_PTR(ret);
}
+static void __destroy_hw_context(struct i915_gem_context *ctx,
+ struct drm_i915_file_private *file_priv)
+{
+ idr_remove(&file_priv->context_idr, ctx->user_handle);
+ context_close(ctx);
+}
+
/**
* The default context needs to exist per ring that uses contexts. It stores the
* context state of the GPU for applications that don't utilize HW contexts, as
if (IS_ERR(ppgtt)) {
DRM_DEBUG_DRIVER("PPGTT setup failed (%ld)\n",
PTR_ERR(ppgtt));
- idr_remove(&file_priv->context_idr, ctx->user_handle);
- context_close(ctx);
+ __destroy_hw_context(ctx, file_priv);
return ERR_CAST(ppgtt);
}
ctx->ppgtt = ppgtt;
+ ctx->desc_template = default_desc_template(dev_priv, ppgtt);
}
trace_i915_context_create(ctx);
i915_gem_context_set_closed(ctx); /* not user accessible */
i915_gem_context_clear_bannable(ctx);
i915_gem_context_set_force_single_submission(ctx);
- ctx->ring_size = 512 * PAGE_SIZE; /* Max ring buffer size */
+ if (!i915.enable_guc_submission)
+ ctx->ring_size = 512 * PAGE_SIZE; /* Max ring buffer size */
GEM_BUG_ON(i915_gem_context_is_kernel(ctx));
out:
return PTR_ERR(ctx);
}
+ /* For easy recognisablity, we want the kernel context to be 0 and then
+ * all user contexts will have non-zero hw_id.
+ */
+ GEM_BUG_ON(ctx->hw_id);
+
i915_gem_context_clear_bannable(ctx);
ctx->priority = I915_PRIORITY_MIN; /* lowest priority; idle task */
dev_priv->kernel_context = ctx;
mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
{
struct drm_i915_private *dev_priv = req->i915;
- struct intel_ring *ring = req->ring;
struct intel_engine_cs *engine = req->engine;
enum intel_engine_id id;
- u32 flags = hw_flags | MI_MM_SPACE_GTT;
+ u32 *cs, flags = hw_flags | MI_MM_SPACE_GTT;
const int num_rings =
/* Use an extended w/a on ivb+ if signalling from other rings */
i915.semaphores ?
INTEL_INFO(dev_priv)->num_rings - 1 :
0;
- int len, ret;
-
- /* w/a: If Flush TLB Invalidation Mode is enabled, driver must do a TLB
- * invalidation prior to MI_SET_CONTEXT. On GEN6 we don't set the value
- * explicitly, so we rely on the value at ring init, stored in
- * itlb_before_ctx_switch.
- */
- if (IS_GEN6(dev_priv)) {
- ret = engine->emit_flush(req, EMIT_INVALIDATE);
- if (ret)
- return ret;
- }
+ int len;
/* These flags are for resource streamer on HSW+ */
if (IS_HASWELL(dev_priv) || INTEL_GEN(dev_priv) >= 8)
if (INTEL_GEN(dev_priv) >= 7)
len += 2 + (num_rings ? 4*num_rings + 6 : 0);
- ret = intel_ring_begin(req, len);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, len);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
/* WaProgramMiArbOnOffAroundMiSetContext:ivb,vlv,hsw,bdw,chv */
if (INTEL_GEN(dev_priv) >= 7) {
- intel_ring_emit(ring, MI_ARB_ON_OFF | MI_ARB_DISABLE);
+ *cs++ = MI_ARB_ON_OFF | MI_ARB_DISABLE;
if (num_rings) {
struct intel_engine_cs *signaller;
- intel_ring_emit(ring,
- MI_LOAD_REGISTER_IMM(num_rings));
+ *cs++ = MI_LOAD_REGISTER_IMM(num_rings);
for_each_engine(signaller, dev_priv, id) {
if (signaller == engine)
continue;
- intel_ring_emit_reg(ring,
- RING_PSMI_CTL(signaller->mmio_base));
- intel_ring_emit(ring,
- _MASKED_BIT_ENABLE(GEN6_PSMI_SLEEP_MSG_DISABLE));
+ *cs++ = i915_mmio_reg_offset(
+ RING_PSMI_CTL(signaller->mmio_base));
+ *cs++ = _MASKED_BIT_ENABLE(
+ GEN6_PSMI_SLEEP_MSG_DISABLE);
}
}
}
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_emit(ring, MI_SET_CONTEXT);
- intel_ring_emit(ring,
- i915_ggtt_offset(req->ctx->engine[RCS].state) | flags);
+ *cs++ = MI_NOOP;
+ *cs++ = MI_SET_CONTEXT;
+ *cs++ = i915_ggtt_offset(req->ctx->engine[RCS].state) | flags;
/*
* w/a: MI_SET_CONTEXT must always be followed by MI_NOOP
* WaMiSetContext_Hang:snb,ivb,vlv
*/
- intel_ring_emit(ring, MI_NOOP);
+ *cs++ = MI_NOOP;
if (INTEL_GEN(dev_priv) >= 7) {
if (num_rings) {
struct intel_engine_cs *signaller;
i915_reg_t last_reg = {}; /* keep gcc quiet */
- intel_ring_emit(ring,
- MI_LOAD_REGISTER_IMM(num_rings));
+ *cs++ = MI_LOAD_REGISTER_IMM(num_rings);
for_each_engine(signaller, dev_priv, id) {
if (signaller == engine)
continue;
last_reg = RING_PSMI_CTL(signaller->mmio_base);
- intel_ring_emit_reg(ring, last_reg);
- intel_ring_emit(ring,
- _MASKED_BIT_DISABLE(GEN6_PSMI_SLEEP_MSG_DISABLE));
+ *cs++ = i915_mmio_reg_offset(last_reg);
+ *cs++ = _MASKED_BIT_DISABLE(
+ GEN6_PSMI_SLEEP_MSG_DISABLE);
}
/* Insert a delay before the next switch! */
- intel_ring_emit(ring,
- MI_STORE_REGISTER_MEM |
- MI_SRM_LRM_GLOBAL_GTT);
- intel_ring_emit_reg(ring, last_reg);
- intel_ring_emit(ring,
- i915_ggtt_offset(engine->scratch));
- intel_ring_emit(ring, MI_NOOP);
+ *cs++ = MI_STORE_REGISTER_MEM | MI_SRM_LRM_GLOBAL_GTT;
+ *cs++ = i915_mmio_reg_offset(last_reg);
+ *cs++ = i915_ggtt_offset(engine->scratch);
+ *cs++ = MI_NOOP;
}
- intel_ring_emit(ring, MI_ARB_ON_OFF | MI_ARB_ENABLE);
+ *cs++ = MI_ARB_ON_OFF | MI_ARB_ENABLE;
}
- intel_ring_advance(ring);
+ intel_ring_advance(req, cs);
- return ret;
+ return 0;
}
static int remap_l3(struct drm_i915_gem_request *req, int slice)
{
- u32 *remap_info = req->i915->l3_parity.remap_info[slice];
- struct intel_ring *ring = req->ring;
- int i, ret;
+ u32 *cs, *remap_info = req->i915->l3_parity.remap_info[slice];
+ int i;
if (!remap_info)
return 0;
- ret = intel_ring_begin(req, GEN7_L3LOG_SIZE/4 * 2 + 2);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, GEN7_L3LOG_SIZE/4 * 2 + 2);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
/*
* Note: We do not worry about the concurrent register cacheline hang
* here because no other code should access these registers other than
* at initialization time.
*/
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(GEN7_L3LOG_SIZE/4));
+ *cs++ = MI_LOAD_REGISTER_IMM(GEN7_L3LOG_SIZE/4);
for (i = 0; i < GEN7_L3LOG_SIZE/4; i++) {
- intel_ring_emit_reg(ring, GEN7_L3LOG(slice, i));
- intel_ring_emit(ring, remap_info[i]);
+ *cs++ = i915_mmio_reg_offset(GEN7_L3LOG(slice, i));
+ *cs++ = remap_info[i];
}
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
}
ret = i915_switch_context(req);
- i915_add_request_no_flush(req);
+ i915_add_request(req);
if (ret)
return ret;
}
return PTR_ERR(ctx);
}
- idr_remove(&file_priv->context_idr, ctx->user_handle);
- context_close(ctx);
+ __destroy_hw_context(ctx, file_priv);
mutex_unlock(&dev->struct_mutex);
DRM_DEBUG("HW context %d destroyed\n", args->ctx_id);
return 0;
}
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/mock_context.c"
+#include "selftests/i915_gem_context.c"
+#endif
*/
int priority;
- /** ggtt_alignment: alignment restriction for context objects */
- u32 ggtt_alignment;
/** ggtt_offset_bias: placement restriction for context objects */
u32 ggtt_offset_bias;
return ERR_PTR(ret);
}
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/mock_dmabuf.c"
+#include "selftests/i915_gem_dmabuf.c"
+#endif
int ret = 0;
lockdep_assert_held(&vm->i915->drm.struct_mutex);
+ GEM_BUG_ON(!IS_ALIGNED(start, I915_GTT_PAGE_SIZE));
+ GEM_BUG_ON(!IS_ALIGNED(end, I915_GTT_PAGE_SIZE));
+
trace_i915_gem_evict_node(vm, target, flags);
/* Retire before we search the active list. Although we have
check_color = vm->mm.color_adjust;
if (check_color) {
/* Expand search to cover neighbouring guard pages (or lack!) */
- if (start > vm->start)
+ if (start)
start -= I915_GTT_PAGE_SIZE;
- if (end < vm->start + vm->total)
- end += I915_GTT_PAGE_SIZE;
+
+ /* Always look at the page afterwards to avoid the end-of-GTT */
+ end += I915_GTT_PAGE_SIZE;
}
+ GEM_BUG_ON(start >= end);
drm_mm_for_each_node_in_range(node, &vm->mm, start, end) {
/* If we find any non-objects (!vma), we cannot evict them */
break;
}
+ GEM_BUG_ON(!node->allocated);
vma = container_of(node, typeof(*vma), node);
/* If we are using coloring to insert guard pages between
return 0;
}
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/i915_gem_evict.c"
+#endif
#include <linux/dma_remapping.h>
#include <linux/reservation.h>
+#include <linux/sync_file.h>
#include <linux/uaccess.h>
#include <drm/drmP.h>
#include <drm/i915_drm.h>
#include "i915_drv.h"
+#include "i915_gem_clflush.h"
#include "i915_trace.h"
#include "intel_drv.h"
#include "intel_frontbuffer.h"
list_for_each_entry(vma, vmas, exec_list) {
struct drm_i915_gem_object *obj = vma->obj;
+ if (vma->exec_entry->flags & EXEC_OBJECT_ASYNC)
+ continue;
+
+ if (obj->base.write_domain & I915_GEM_DOMAIN_CPU) {
+ i915_gem_clflush_object(obj, 0);
+ obj->base.write_domain = 0;
+ }
+
ret = i915_gem_request_await_object
(req, obj, obj->base.pending_write_domain);
if (ret)
return ret;
-
- if (obj->base.write_domain & I915_GEM_DOMAIN_CPU)
- i915_gem_clflush_object(obj, false);
}
/* Unconditionally flush any chipset caches (for streaming writes). */
* handle an error right now. Worst case should be missed
* synchronisation leading to rendering corruption.
*/
- ww_mutex_lock(&resv->lock, NULL);
+ reservation_object_lock(resv, NULL);
if (flags & EXEC_OBJECT_WRITE)
reservation_object_add_excl_fence(resv, &req->fence);
else if (reservation_object_reserve_shared(resv) == 0)
reservation_object_add_shared_fence(resv, &req->fence);
- ww_mutex_unlock(&resv->lock);
+ reservation_object_unlock(resv);
}
static void
list_for_each_entry(vma, vmas, exec_list) {
struct drm_i915_gem_object *obj = vma->obj;
- u32 old_read = obj->base.read_domains;
- u32 old_write = obj->base.write_domain;
obj->base.write_domain = obj->base.pending_write_domain;
if (obj->base.write_domain)
i915_vma_move_to_active(vma, req, vma->exec_entry->flags);
eb_export_fence(obj, req, vma->exec_entry->flags);
- trace_i915_gem_object_change_domain(obj, old_read, old_write);
}
}
static int
i915_reset_gen7_sol_offsets(struct drm_i915_gem_request *req)
{
- struct intel_ring *ring = req->ring;
- int ret, i;
+ u32 *cs;
+ int i;
if (!IS_GEN7(req->i915) || req->engine->id != RCS) {
DRM_DEBUG("sol reset is gen7/rcs only\n");
return -EINVAL;
}
- ret = intel_ring_begin(req, 4 * 3);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4 * 3);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
for (i = 0; i < 4; i++) {
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ring, GEN7_SO_WRITE_OFFSET(i));
- intel_ring_emit(ring, 0);
+ *cs++ = MI_LOAD_REGISTER_IMM(1);
+ *cs++ = i915_mmio_reg_offset(GEN7_SO_WRITE_OFFSET(i));
+ *cs++ = 0;
}
- intel_ring_advance(ring);
+ intel_ring_advance(req, cs);
return 0;
}
return vma;
}
+static void
+add_to_client(struct drm_i915_gem_request *req,
+ struct drm_file *file)
+{
+ req->file_priv = file->driver_priv;
+ list_add_tail(&req->client_link, &req->file_priv->mm.request_list);
+}
+
static int
execbuf_submit(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
if (ret)
return ret;
- trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
-
i915_gem_execbuffer_move_to_active(vmas, params->request);
return 0;
struct i915_execbuffer_params *params = ¶ms_master;
const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
u32 dispatch_flags;
+ struct dma_fence *in_fence = NULL;
+ struct sync_file *out_fence = NULL;
+ int out_fence_fd = -1;
int ret;
bool need_relocs;
dispatch_flags |= I915_DISPATCH_RS;
}
+ if (args->flags & I915_EXEC_FENCE_IN) {
+ in_fence = sync_file_get_fence(lower_32_bits(args->rsvd2));
+ if (!in_fence)
+ return -EINVAL;
+ }
+
+ if (args->flags & I915_EXEC_FENCE_OUT) {
+ out_fence_fd = get_unused_fd_flags(O_CLOEXEC);
+ if (out_fence_fd < 0) {
+ ret = out_fence_fd;
+ goto err_in_fence;
+ }
+ }
+
/* Take a local wakeref for preparing to dispatch the execbuf as
* we expect to access the hardware fairly frequently in the
* process. Upon first dispatch, we acquire another prolonged
goto err_batch_unpin;
}
+ if (in_fence) {
+ ret = i915_gem_request_await_dma_fence(params->request,
+ in_fence);
+ if (ret < 0)
+ goto err_request;
+ }
+
+ if (out_fence_fd != -1) {
+ out_fence = sync_file_create(¶ms->request->fence);
+ if (!out_fence) {
+ ret = -ENOMEM;
+ goto err_request;
+ }
+ }
+
/* Whilst this request exists, batch_obj will be on the
* active_list, and so will hold the active reference. Only when this
* request is retired will the the batch_obj be moved onto the
*/
params->request->batch = params->batch;
- ret = i915_gem_request_add_to_client(params->request, file);
- if (ret)
- goto err_request;
-
/*
* Save assorted stuff away to pass through to *_submission().
* NB: This data should be 'persistent' and not local as it will
params->dispatch_flags = dispatch_flags;
params->ctx = ctx;
+ trace_i915_gem_request_queue(params->request, dispatch_flags);
+
ret = execbuf_submit(params, args, &eb->vmas);
err_request:
__i915_add_request(params->request, ret == 0);
+ add_to_client(params->request, file);
+
+ if (out_fence) {
+ if (ret == 0) {
+ fd_install(out_fence_fd, out_fence->file);
+ args->rsvd2 &= GENMASK_ULL(0, 31); /* keep in-fence */
+ args->rsvd2 |= (u64)out_fence_fd << 32;
+ out_fence_fd = -1;
+ } else {
+ fput(out_fence->file);
+ }
+ }
err_batch_unpin:
/*
/* intel_gpu_busy should also get a ref, so it will free when the device
* is really idle. */
intel_runtime_pm_put(dev_priv);
+ if (out_fence_fd != -1)
+ put_unused_fd(out_fence_fd);
+err_in_fence:
+ dma_fence_put(in_fence);
return ret;
}
return -EINVAL;
}
- if (args->rsvd2 != 0) {
- DRM_DEBUG("dirty rvsd2 field\n");
- return -EINVAL;
- }
-
exec2_list = drm_malloc_gfp(args->buffer_count,
sizeof(*exec2_list),
GFP_TEMPORARY);
list_move(&fence->link, &fence->i915->mm.fence_list);
}
- fence_write(fence, vma);
+ /* We only need to update the register itself if the device is awake.
+ * If the device is currently powered down, we will defer the write
+ * to the runtime resume, see i915_gem_restore_fences().
+ */
+ if (intel_runtime_pm_get_if_in_use(fence->i915)) {
+ fence_write(fence, vma);
+ intel_runtime_pm_put(fence->i915);
+ }
if (vma) {
if (fence->vma != vma) {
{
struct drm_i915_fence_reg *fence = vma->fence;
- assert_rpm_wakelock_held(vma->vm->i915);
-
if (!fence)
return 0;
*
*/
+#include <linux/slab.h> /* fault-inject.h is not standalone! */
+
+#include <linux/fault-inject.h>
#include <linux/log2.h>
#include <linux/random.h>
#include <linux/seq_file.h>
enum i915_cache_level cache_level,
u32 unused)
{
- u32 pte_flags = 0;
+ u32 pte_flags;
+ int ret;
+
+ ret = vma->vm->allocate_va_range(vma->vm, vma->node.start, vma->size);
+ if (ret)
+ return ret;
vma->pages = vma->obj->mm.pages;
/* Currently applicable only to VLV */
+ pte_flags = 0;
if (vma->obj->gt_ro)
pte_flags |= PTE_READ_ONLY;
static void ppgtt_unbind_vma(struct i915_vma *vma)
{
- vma->vm->clear_range(vma->vm,
- vma->node.start,
- vma->size);
+ vma->vm->clear_range(vma->vm, vma->node.start, vma->size);
}
static gen8_pte_t gen8_pte_encode(dma_addr_t addr,
return pte;
}
-static int __setup_page_dma(struct drm_i915_private *dev_priv,
- struct i915_page_dma *p, gfp_t flags)
+static struct page *vm_alloc_page(struct i915_address_space *vm, gfp_t gfp)
{
- struct device *kdev = &dev_priv->drm.pdev->dev;
+ struct page *page;
- p->page = alloc_page(flags);
- if (!p->page)
- return -ENOMEM;
+ if (I915_SELFTEST_ONLY(should_fail(&vm->fault_attr, 1)))
+ i915_gem_shrink_all(vm->i915);
- p->daddr = dma_map_page(kdev,
- p->page, 0, PAGE_SIZE, PCI_DMA_BIDIRECTIONAL);
+ if (vm->free_pages.nr)
+ return vm->free_pages.pages[--vm->free_pages.nr];
- if (dma_mapping_error(kdev, p->daddr)) {
- __free_page(p->page);
- return -EINVAL;
- }
+ page = alloc_page(gfp);
+ if (!page)
+ return NULL;
- return 0;
+ if (vm->pt_kmap_wc)
+ set_pages_array_wc(&page, 1);
+
+ return page;
}
-static int setup_page_dma(struct drm_i915_private *dev_priv,
- struct i915_page_dma *p)
+static void vm_free_pages_release(struct i915_address_space *vm)
{
- return __setup_page_dma(dev_priv, p, I915_GFP_DMA);
+ GEM_BUG_ON(!pagevec_count(&vm->free_pages));
+
+ if (vm->pt_kmap_wc)
+ set_pages_array_wb(vm->free_pages.pages,
+ pagevec_count(&vm->free_pages));
+
+ __pagevec_release(&vm->free_pages);
}
-static void cleanup_page_dma(struct drm_i915_private *dev_priv,
- struct i915_page_dma *p)
+static void vm_free_page(struct i915_address_space *vm, struct page *page)
{
- struct pci_dev *pdev = dev_priv->drm.pdev;
+ if (!pagevec_add(&vm->free_pages, page))
+ vm_free_pages_release(vm);
+}
- if (WARN_ON(!p->page))
- return;
+static int __setup_page_dma(struct i915_address_space *vm,
+ struct i915_page_dma *p,
+ gfp_t gfp)
+{
+ p->page = vm_alloc_page(vm, gfp | __GFP_NOWARN | __GFP_NORETRY);
+ if (unlikely(!p->page))
+ return -ENOMEM;
- dma_unmap_page(&pdev->dev, p->daddr, PAGE_SIZE, PCI_DMA_BIDIRECTIONAL);
- __free_page(p->page);
- memset(p, 0, sizeof(*p));
+ p->daddr = dma_map_page(vm->dma, p->page, 0, PAGE_SIZE,
+ PCI_DMA_BIDIRECTIONAL);
+ if (unlikely(dma_mapping_error(vm->dma, p->daddr))) {
+ vm_free_page(vm, p->page);
+ return -ENOMEM;
+ }
+
+ return 0;
}
-static void *kmap_page_dma(struct i915_page_dma *p)
+static int setup_page_dma(struct i915_address_space *vm,
+ struct i915_page_dma *p)
{
- return kmap_atomic(p->page);
+ return __setup_page_dma(vm, p, I915_GFP_DMA);
}
-/* We use the flushing unmap only with ppgtt structures:
- * page directories, page tables and scratch pages.
- */
-static void kunmap_page_dma(struct drm_i915_private *dev_priv, void *vaddr)
+static void cleanup_page_dma(struct i915_address_space *vm,
+ struct i915_page_dma *p)
{
- /* There are only few exceptions for gen >=6. chv and bxt.
- * And we are not sure about the latter so play safe for now.
- */
- if (IS_CHERRYVIEW(dev_priv) || IS_GEN9_LP(dev_priv))
- drm_clflush_virt_range(vaddr, PAGE_SIZE);
-
- kunmap_atomic(vaddr);
+ dma_unmap_page(vm->dma, p->daddr, PAGE_SIZE, PCI_DMA_BIDIRECTIONAL);
+ vm_free_page(vm, p->page);
}
-#define kmap_px(px) kmap_page_dma(px_base(px))
-#define kunmap_px(ppgtt, vaddr) \
- kunmap_page_dma((ppgtt)->base.i915, (vaddr))
+#define kmap_atomic_px(px) kmap_atomic(px_base(px)->page)
-#define setup_px(dev_priv, px) setup_page_dma((dev_priv), px_base(px))
-#define cleanup_px(dev_priv, px) cleanup_page_dma((dev_priv), px_base(px))
-#define fill_px(dev_priv, px, v) fill_page_dma((dev_priv), px_base(px), (v))
-#define fill32_px(dev_priv, px, v) \
- fill_page_dma_32((dev_priv), px_base(px), (v))
+#define setup_px(vm, px) setup_page_dma((vm), px_base(px))
+#define cleanup_px(vm, px) cleanup_page_dma((vm), px_base(px))
+#define fill_px(ppgtt, px, v) fill_page_dma((vm), px_base(px), (v))
+#define fill32_px(ppgtt, px, v) fill_page_dma_32((vm), px_base(px), (v))
-static void fill_page_dma(struct drm_i915_private *dev_priv,
- struct i915_page_dma *p, const uint64_t val)
+static void fill_page_dma(struct i915_address_space *vm,
+ struct i915_page_dma *p,
+ const u64 val)
{
+ u64 * const vaddr = kmap_atomic(p->page);
int i;
- uint64_t * const vaddr = kmap_page_dma(p);
for (i = 0; i < 512; i++)
vaddr[i] = val;
- kunmap_page_dma(dev_priv, vaddr);
+ kunmap_atomic(vaddr);
}
-static void fill_page_dma_32(struct drm_i915_private *dev_priv,
- struct i915_page_dma *p, const uint32_t val32)
+static void fill_page_dma_32(struct i915_address_space *vm,
+ struct i915_page_dma *p,
+ const u32 v)
{
- uint64_t v = val32;
-
- v = v << 32 | val32;
-
- fill_page_dma(dev_priv, p, v);
+ fill_page_dma(vm, p, (u64)v << 32 | v);
}
static int
-setup_scratch_page(struct drm_i915_private *dev_priv,
- struct i915_page_dma *scratch,
- gfp_t gfp)
+setup_scratch_page(struct i915_address_space *vm, gfp_t gfp)
{
- return __setup_page_dma(dev_priv, scratch, gfp | __GFP_ZERO);
+ return __setup_page_dma(vm, &vm->scratch_page, gfp | __GFP_ZERO);
}
-static void cleanup_scratch_page(struct drm_i915_private *dev_priv,
- struct i915_page_dma *scratch)
+static void cleanup_scratch_page(struct i915_address_space *vm)
{
- cleanup_page_dma(dev_priv, scratch);
+ cleanup_page_dma(vm, &vm->scratch_page);
}
-static struct i915_page_table *alloc_pt(struct drm_i915_private *dev_priv)
+static struct i915_page_table *alloc_pt(struct i915_address_space *vm)
{
struct i915_page_table *pt;
- const size_t count = INTEL_GEN(dev_priv) >= 8 ? GEN8_PTES : GEN6_PTES;
- int ret = -ENOMEM;
- pt = kzalloc(sizeof(*pt), GFP_KERNEL);
- if (!pt)
+ pt = kmalloc(sizeof(*pt), GFP_KERNEL | __GFP_NOWARN);
+ if (unlikely(!pt))
return ERR_PTR(-ENOMEM);
- pt->used_ptes = kcalloc(BITS_TO_LONGS(count), sizeof(*pt->used_ptes),
- GFP_KERNEL);
-
- if (!pt->used_ptes)
- goto fail_bitmap;
-
- ret = setup_px(dev_priv, pt);
- if (ret)
- goto fail_page_m;
+ if (unlikely(setup_px(vm, pt))) {
+ kfree(pt);
+ return ERR_PTR(-ENOMEM);
+ }
+ pt->used_ptes = 0;
return pt;
-
-fail_page_m:
- kfree(pt->used_ptes);
-fail_bitmap:
- kfree(pt);
-
- return ERR_PTR(ret);
}
-static void free_pt(struct drm_i915_private *dev_priv,
- struct i915_page_table *pt)
+static void free_pt(struct i915_address_space *vm, struct i915_page_table *pt)
{
- cleanup_px(dev_priv, pt);
- kfree(pt->used_ptes);
+ cleanup_px(vm, pt);
kfree(pt);
}
static void gen8_initialize_pt(struct i915_address_space *vm,
struct i915_page_table *pt)
{
- gen8_pte_t scratch_pte;
-
- scratch_pte = gen8_pte_encode(vm->scratch_page.daddr,
- I915_CACHE_LLC);
-
- fill_px(vm->i915, pt, scratch_pte);
+ fill_px(vm, pt,
+ gen8_pte_encode(vm->scratch_page.daddr, I915_CACHE_LLC));
}
static void gen6_initialize_pt(struct i915_address_space *vm,
struct i915_page_table *pt)
{
- gen6_pte_t scratch_pte;
-
- WARN_ON(vm->scratch_page.daddr == 0);
-
- scratch_pte = vm->pte_encode(vm->scratch_page.daddr,
- I915_CACHE_LLC, 0);
-
- fill32_px(vm->i915, pt, scratch_pte);
+ fill32_px(vm, pt,
+ vm->pte_encode(vm->scratch_page.daddr, I915_CACHE_LLC, 0));
}
-static struct i915_page_directory *alloc_pd(struct drm_i915_private *dev_priv)
+static struct i915_page_directory *alloc_pd(struct i915_address_space *vm)
{
struct i915_page_directory *pd;
- int ret = -ENOMEM;
- pd = kzalloc(sizeof(*pd), GFP_KERNEL);
- if (!pd)
+ pd = kzalloc(sizeof(*pd), GFP_KERNEL | __GFP_NOWARN);
+ if (unlikely(!pd))
return ERR_PTR(-ENOMEM);
- pd->used_pdes = kcalloc(BITS_TO_LONGS(I915_PDES),
- sizeof(*pd->used_pdes), GFP_KERNEL);
- if (!pd->used_pdes)
- goto fail_bitmap;
-
- ret = setup_px(dev_priv, pd);
- if (ret)
- goto fail_page_m;
+ if (unlikely(setup_px(vm, pd))) {
+ kfree(pd);
+ return ERR_PTR(-ENOMEM);
+ }
+ pd->used_pdes = 0;
return pd;
-
-fail_page_m:
- kfree(pd->used_pdes);
-fail_bitmap:
- kfree(pd);
-
- return ERR_PTR(ret);
}
-static void free_pd(struct drm_i915_private *dev_priv,
+static void free_pd(struct i915_address_space *vm,
struct i915_page_directory *pd)
{
- if (px_page(pd)) {
- cleanup_px(dev_priv, pd);
- kfree(pd->used_pdes);
- kfree(pd);
- }
+ cleanup_px(vm, pd);
+ kfree(pd);
}
static void gen8_initialize_pd(struct i915_address_space *vm,
struct i915_page_directory *pd)
{
- gen8_pde_t scratch_pde;
-
- scratch_pde = gen8_pde_encode(px_dma(vm->scratch_pt), I915_CACHE_LLC);
+ unsigned int i;
- fill_px(vm->i915, pd, scratch_pde);
+ fill_px(vm, pd,
+ gen8_pde_encode(px_dma(vm->scratch_pt), I915_CACHE_LLC));
+ for (i = 0; i < I915_PDES; i++)
+ pd->page_table[i] = vm->scratch_pt;
}
-static int __pdp_init(struct drm_i915_private *dev_priv,
+static int __pdp_init(struct i915_address_space *vm,
struct i915_page_directory_pointer *pdp)
{
- size_t pdpes = I915_PDPES_PER_PDP(dev_priv);
+ const unsigned int pdpes = i915_pdpes_per_pdp(vm);
+ unsigned int i;
- pdp->used_pdpes = kcalloc(BITS_TO_LONGS(pdpes),
- sizeof(unsigned long),
- GFP_KERNEL);
- if (!pdp->used_pdpes)
+ pdp->page_directory = kmalloc_array(pdpes, sizeof(*pdp->page_directory),
+ GFP_KERNEL | __GFP_NOWARN);
+ if (unlikely(!pdp->page_directory))
return -ENOMEM;
- pdp->page_directory = kcalloc(pdpes, sizeof(*pdp->page_directory),
- GFP_KERNEL);
- if (!pdp->page_directory) {
- kfree(pdp->used_pdpes);
- /* the PDP might be the statically allocated top level. Keep it
- * as clean as possible */
- pdp->used_pdpes = NULL;
- return -ENOMEM;
- }
+ for (i = 0; i < pdpes; i++)
+ pdp->page_directory[i] = vm->scratch_pd;
return 0;
}
static void __pdp_fini(struct i915_page_directory_pointer *pdp)
{
- kfree(pdp->used_pdpes);
kfree(pdp->page_directory);
pdp->page_directory = NULL;
}
-static struct
-i915_page_directory_pointer *alloc_pdp(struct drm_i915_private *dev_priv)
+static inline bool use_4lvl(const struct i915_address_space *vm)
+{
+ return i915_vm_is_48bit(vm);
+}
+
+static struct i915_page_directory_pointer *
+alloc_pdp(struct i915_address_space *vm)
{
struct i915_page_directory_pointer *pdp;
int ret = -ENOMEM;
- WARN_ON(!USES_FULL_48BIT_PPGTT(dev_priv));
+ WARN_ON(!use_4lvl(vm));
pdp = kzalloc(sizeof(*pdp), GFP_KERNEL);
if (!pdp)
return ERR_PTR(-ENOMEM);
- ret = __pdp_init(dev_priv, pdp);
+ ret = __pdp_init(vm, pdp);
if (ret)
goto fail_bitmap;
- ret = setup_px(dev_priv, pdp);
+ ret = setup_px(vm, pdp);
if (ret)
goto fail_page_m;
return ERR_PTR(ret);
}
-static void free_pdp(struct drm_i915_private *dev_priv,
+static void free_pdp(struct i915_address_space *vm,
struct i915_page_directory_pointer *pdp)
{
__pdp_fini(pdp);
- if (USES_FULL_48BIT_PPGTT(dev_priv)) {
- cleanup_px(dev_priv, pdp);
- kfree(pdp);
- }
+
+ if (!use_4lvl(vm))
+ return;
+
+ cleanup_px(vm, pdp);
+ kfree(pdp);
}
static void gen8_initialize_pdp(struct i915_address_space *vm,
scratch_pdpe = gen8_pdpe_encode(px_dma(vm->scratch_pd), I915_CACHE_LLC);
- fill_px(vm->i915, pdp, scratch_pdpe);
+ fill_px(vm, pdp, scratch_pdpe);
}
static void gen8_initialize_pml4(struct i915_address_space *vm,
struct i915_pml4 *pml4)
{
- gen8_ppgtt_pml4e_t scratch_pml4e;
-
- scratch_pml4e = gen8_pml4e_encode(px_dma(vm->scratch_pdp),
- I915_CACHE_LLC);
-
- fill_px(vm->i915, pml4, scratch_pml4e);
-}
-
-static void
-gen8_setup_pdpe(struct i915_hw_ppgtt *ppgtt,
- struct i915_page_directory_pointer *pdp,
- struct i915_page_directory *pd,
- int index)
-{
- gen8_ppgtt_pdpe_t *page_directorypo;
-
- if (!USES_FULL_48BIT_PPGTT(to_i915(ppgtt->base.dev)))
- return;
-
- page_directorypo = kmap_px(pdp);
- page_directorypo[index] = gen8_pdpe_encode(px_dma(pd), I915_CACHE_LLC);
- kunmap_px(ppgtt, page_directorypo);
-}
-
-static void
-gen8_setup_pml4e(struct i915_hw_ppgtt *ppgtt,
- struct i915_pml4 *pml4,
- struct i915_page_directory_pointer *pdp,
- int index)
-{
- gen8_ppgtt_pml4e_t *pagemap = kmap_px(pml4);
+ unsigned int i;
- WARN_ON(!USES_FULL_48BIT_PPGTT(to_i915(ppgtt->base.dev)));
- pagemap[index] = gen8_pml4e_encode(px_dma(pdp), I915_CACHE_LLC);
- kunmap_px(ppgtt, pagemap);
+ fill_px(vm, pml4,
+ gen8_pml4e_encode(px_dma(vm->scratch_pdp), I915_CACHE_LLC));
+ for (i = 0; i < GEN8_PML4ES_PER_PML4; i++)
+ pml4->pdps[i] = vm->scratch_pdp;
}
/* Broadwell Page Directory Pointer Descriptors */
unsigned entry,
dma_addr_t addr)
{
- struct intel_ring *ring = req->ring;
struct intel_engine_cs *engine = req->engine;
- int ret;
+ u32 *cs;
BUG_ON(entry >= 4);
- ret = intel_ring_begin(req, 6);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 6);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ring, GEN8_RING_PDP_UDW(engine, entry));
- intel_ring_emit(ring, upper_32_bits(addr));
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ring, GEN8_RING_PDP_LDW(engine, entry));
- intel_ring_emit(ring, lower_32_bits(addr));
- intel_ring_advance(ring);
+ *cs++ = MI_LOAD_REGISTER_IMM(1);
+ *cs++ = i915_mmio_reg_offset(GEN8_RING_PDP_UDW(engine, entry));
+ *cs++ = upper_32_bits(addr);
+ *cs++ = MI_LOAD_REGISTER_IMM(1);
+ *cs++ = i915_mmio_reg_offset(GEN8_RING_PDP_LDW(engine, entry));
+ *cs++ = lower_32_bits(addr);
+ intel_ring_advance(req, cs);
return 0;
}
-static int gen8_legacy_mm_switch(struct i915_hw_ppgtt *ppgtt,
- struct drm_i915_gem_request *req)
+static int gen8_mm_switch_3lvl(struct i915_hw_ppgtt *ppgtt,
+ struct drm_i915_gem_request *req)
{
int i, ret;
- for (i = GEN8_LEGACY_PDPES - 1; i >= 0; i--) {
+ for (i = GEN8_3LVL_PDPES - 1; i >= 0; i--) {
const dma_addr_t pd_daddr = i915_page_dir_dma_addr(ppgtt, i);
ret = gen8_write_pdp(req, i, pd_daddr);
return 0;
}
-static int gen8_48b_mm_switch(struct i915_hw_ppgtt *ppgtt,
- struct drm_i915_gem_request *req)
+static int gen8_mm_switch_4lvl(struct i915_hw_ppgtt *ppgtt,
+ struct drm_i915_gem_request *req)
{
return gen8_write_pdp(req, 0, px_dma(&ppgtt->pml4));
}
*/
static bool gen8_ppgtt_clear_pt(struct i915_address_space *vm,
struct i915_page_table *pt,
- uint64_t start,
- uint64_t length)
+ u64 start, u64 length)
{
- struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
unsigned int num_entries = gen8_pte_count(start, length);
unsigned int pte = gen8_pte_index(start);
unsigned int pte_end = pte + num_entries;
- gen8_pte_t *pt_vaddr;
- gen8_pte_t scratch_pte = gen8_pte_encode(vm->scratch_page.daddr,
- I915_CACHE_LLC);
+ const gen8_pte_t scratch_pte =
+ gen8_pte_encode(vm->scratch_page.daddr, I915_CACHE_LLC);
+ gen8_pte_t *vaddr;
- if (WARN_ON(!px_page(pt)))
- return false;
+ GEM_BUG_ON(num_entries > pt->used_ptes);
- GEM_BUG_ON(pte_end > GEN8_PTES);
+ pt->used_ptes -= num_entries;
+ if (!pt->used_ptes)
+ return true;
- bitmap_clear(pt->used_ptes, pte, num_entries);
- if (USES_FULL_PPGTT(vm->i915)) {
- if (bitmap_empty(pt->used_ptes, GEN8_PTES))
- return true;
- }
+ vaddr = kmap_atomic_px(pt);
+ while (pte < pte_end)
+ vaddr[pte++] = scratch_pte;
+ kunmap_atomic(vaddr);
- pt_vaddr = kmap_px(pt);
+ return false;
+}
- while (pte < pte_end)
- pt_vaddr[pte++] = scratch_pte;
+static void gen8_ppgtt_set_pde(struct i915_address_space *vm,
+ struct i915_page_directory *pd,
+ struct i915_page_table *pt,
+ unsigned int pde)
+{
+ gen8_pde_t *vaddr;
- kunmap_px(ppgtt, pt_vaddr);
+ pd->page_table[pde] = pt;
- return false;
+ vaddr = kmap_atomic_px(pd);
+ vaddr[pde] = gen8_pde_encode(px_dma(pt), I915_CACHE_LLC);
+ kunmap_atomic(vaddr);
}
-/* Removes entries from a single page dir, releasing it if it's empty.
- * Caller can use the return value to update higher-level entries
- */
static bool gen8_ppgtt_clear_pd(struct i915_address_space *vm,
struct i915_page_directory *pd,
- uint64_t start,
- uint64_t length)
+ u64 start, u64 length)
{
- struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
struct i915_page_table *pt;
- uint64_t pde;
- gen8_pde_t *pde_vaddr;
- gen8_pde_t scratch_pde = gen8_pde_encode(px_dma(vm->scratch_pt),
- I915_CACHE_LLC);
+ u32 pde;
gen8_for_each_pde(pt, pd, start, length, pde) {
- if (WARN_ON(!pd->page_table[pde]))
- break;
-
- if (gen8_ppgtt_clear_pt(vm, pt, start, length)) {
- __clear_bit(pde, pd->used_pdes);
- pde_vaddr = kmap_px(pd);
- pde_vaddr[pde] = scratch_pde;
- kunmap_px(ppgtt, pde_vaddr);
- free_pt(vm->i915, pt);
- }
+ GEM_BUG_ON(pt == vm->scratch_pt);
+
+ if (!gen8_ppgtt_clear_pt(vm, pt, start, length))
+ continue;
+
+ gen8_ppgtt_set_pde(vm, pd, vm->scratch_pt, pde);
+ GEM_BUG_ON(!pd->used_pdes);
+ pd->used_pdes--;
+
+ free_pt(vm, pt);
}
- if (bitmap_empty(pd->used_pdes, I915_PDES))
- return true;
+ return !pd->used_pdes;
+}
- return false;
+static void gen8_ppgtt_set_pdpe(struct i915_address_space *vm,
+ struct i915_page_directory_pointer *pdp,
+ struct i915_page_directory *pd,
+ unsigned int pdpe)
+{
+ gen8_ppgtt_pdpe_t *vaddr;
+
+ pdp->page_directory[pdpe] = pd;
+ if (!use_4lvl(vm))
+ return;
+
+ vaddr = kmap_atomic_px(pdp);
+ vaddr[pdpe] = gen8_pdpe_encode(px_dma(pd), I915_CACHE_LLC);
+ kunmap_atomic(vaddr);
}
/* Removes entries from a single page dir pointer, releasing it if it's empty.
*/
static bool gen8_ppgtt_clear_pdp(struct i915_address_space *vm,
struct i915_page_directory_pointer *pdp,
- uint64_t start,
- uint64_t length)
+ u64 start, u64 length)
{
- struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
struct i915_page_directory *pd;
- uint64_t pdpe;
+ unsigned int pdpe;
gen8_for_each_pdpe(pd, pdp, start, length, pdpe) {
- if (WARN_ON(!pdp->page_directory[pdpe]))
- break;
+ GEM_BUG_ON(pd == vm->scratch_pd);
- if (gen8_ppgtt_clear_pd(vm, pd, start, length)) {
- __clear_bit(pdpe, pdp->used_pdpes);
- gen8_setup_pdpe(ppgtt, pdp, vm->scratch_pd, pdpe);
- free_pd(vm->i915, pd);
- }
+ if (!gen8_ppgtt_clear_pd(vm, pd, start, length))
+ continue;
+
+ gen8_ppgtt_set_pdpe(vm, pdp, vm->scratch_pd, pdpe);
+ GEM_BUG_ON(!pdp->used_pdpes);
+ pdp->used_pdpes--;
+
+ free_pd(vm, pd);
}
- mark_tlbs_dirty(ppgtt);
+ return !pdp->used_pdpes;
+}
- if (bitmap_empty(pdp->used_pdpes, I915_PDPES_PER_PDP(dev_priv)))
- return true;
+static void gen8_ppgtt_clear_3lvl(struct i915_address_space *vm,
+ u64 start, u64 length)
+{
+ gen8_ppgtt_clear_pdp(vm, &i915_vm_to_ppgtt(vm)->pdp, start, length);
+}
- return false;
+static void gen8_ppgtt_set_pml4e(struct i915_pml4 *pml4,
+ struct i915_page_directory_pointer *pdp,
+ unsigned int pml4e)
+{
+ gen8_ppgtt_pml4e_t *vaddr;
+
+ pml4->pdps[pml4e] = pdp;
+
+ vaddr = kmap_atomic_px(pml4);
+ vaddr[pml4e] = gen8_pml4e_encode(px_dma(pdp), I915_CACHE_LLC);
+ kunmap_atomic(vaddr);
}
/* Removes entries from a single pml4.
* This is the top-level structure in 4-level page tables used on gen8+.
* Empty entries are always scratch pml4e.
*/
-static void gen8_ppgtt_clear_pml4(struct i915_address_space *vm,
- struct i915_pml4 *pml4,
- uint64_t start,
- uint64_t length)
+static void gen8_ppgtt_clear_4lvl(struct i915_address_space *vm,
+ u64 start, u64 length)
{
struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
+ struct i915_pml4 *pml4 = &ppgtt->pml4;
struct i915_page_directory_pointer *pdp;
- uint64_t pml4e;
+ unsigned int pml4e;
- GEM_BUG_ON(!USES_FULL_48BIT_PPGTT(vm->i915));
+ GEM_BUG_ON(!use_4lvl(vm));
gen8_for_each_pml4e(pdp, pml4, start, length, pml4e) {
- if (WARN_ON(!pml4->pdps[pml4e]))
- break;
+ GEM_BUG_ON(pdp == vm->scratch_pdp);
- if (gen8_ppgtt_clear_pdp(vm, pdp, start, length)) {
- __clear_bit(pml4e, pml4->used_pml4es);
- gen8_setup_pml4e(ppgtt, pml4, vm->scratch_pdp, pml4e);
- free_pdp(vm->i915, pdp);
- }
+ if (!gen8_ppgtt_clear_pdp(vm, pdp, start, length))
+ continue;
+
+ gen8_ppgtt_set_pml4e(pml4, vm->scratch_pdp, pml4e);
+
+ free_pdp(vm, pdp);
}
}
-static void gen8_ppgtt_clear_range(struct i915_address_space *vm,
- uint64_t start, uint64_t length)
-{
- struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
+struct sgt_dma {
+ struct scatterlist *sg;
+ dma_addr_t dma, max;
+};
- if (USES_FULL_48BIT_PPGTT(vm->i915))
- gen8_ppgtt_clear_pml4(vm, &ppgtt->pml4, start, length);
- else
- gen8_ppgtt_clear_pdp(vm, &ppgtt->pdp, start, length);
+struct gen8_insert_pte {
+ u16 pml4e;
+ u16 pdpe;
+ u16 pde;
+ u16 pte;
+};
+
+static __always_inline struct gen8_insert_pte gen8_insert_pte(u64 start)
+{
+ return (struct gen8_insert_pte) {
+ gen8_pml4e_index(start),
+ gen8_pdpe_index(start),
+ gen8_pde_index(start),
+ gen8_pte_index(start),
+ };
}
-static void
-gen8_ppgtt_insert_pte_entries(struct i915_address_space *vm,
+static __always_inline bool
+gen8_ppgtt_insert_pte_entries(struct i915_hw_ppgtt *ppgtt,
struct i915_page_directory_pointer *pdp,
- struct sg_page_iter *sg_iter,
- uint64_t start,
+ struct sgt_dma *iter,
+ struct gen8_insert_pte *idx,
enum i915_cache_level cache_level)
{
- struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
- gen8_pte_t *pt_vaddr;
- unsigned pdpe = gen8_pdpe_index(start);
- unsigned pde = gen8_pde_index(start);
- unsigned pte = gen8_pte_index(start);
-
- pt_vaddr = NULL;
-
- while (__sg_page_iter_next(sg_iter)) {
- if (pt_vaddr == NULL) {
- struct i915_page_directory *pd = pdp->page_directory[pdpe];
- struct i915_page_table *pt = pd->page_table[pde];
- pt_vaddr = kmap_px(pt);
+ struct i915_page_directory *pd;
+ const gen8_pte_t pte_encode = gen8_pte_encode(0, cache_level);
+ gen8_pte_t *vaddr;
+ bool ret;
+
+ GEM_BUG_ON(idx->pdpe >= i915_pdpes_per_pdp(&ppgtt->base));
+ pd = pdp->page_directory[idx->pdpe];
+ vaddr = kmap_atomic_px(pd->page_table[idx->pde]);
+ do {
+ vaddr[idx->pte] = pte_encode | iter->dma;
+
+ iter->dma += PAGE_SIZE;
+ if (iter->dma >= iter->max) {
+ iter->sg = __sg_next(iter->sg);
+ if (!iter->sg) {
+ ret = false;
+ break;
+ }
+
+ iter->dma = sg_dma_address(iter->sg);
+ iter->max = iter->dma + iter->sg->length;
}
- pt_vaddr[pte] =
- gen8_pte_encode(sg_page_iter_dma_address(sg_iter),
- cache_level);
- if (++pte == GEN8_PTES) {
- kunmap_px(ppgtt, pt_vaddr);
- pt_vaddr = NULL;
- if (++pde == I915_PDES) {
- if (++pdpe == I915_PDPES_PER_PDP(vm->i915))
+ if (++idx->pte == GEN8_PTES) {
+ idx->pte = 0;
+
+ if (++idx->pde == I915_PDES) {
+ idx->pde = 0;
+
+ /* Limited by sg length for 3lvl */
+ if (++idx->pdpe == GEN8_PML4ES_PER_PML4) {
+ idx->pdpe = 0;
+ ret = true;
break;
- pde = 0;
+ }
+
+ GEM_BUG_ON(idx->pdpe >= i915_pdpes_per_pdp(&ppgtt->base));
+ pd = pdp->page_directory[idx->pdpe];
}
- pte = 0;
+
+ kunmap_atomic(vaddr);
+ vaddr = kmap_atomic_px(pd->page_table[idx->pde]);
}
- }
+ } while (1);
+ kunmap_atomic(vaddr);
- if (pt_vaddr)
- kunmap_px(ppgtt, pt_vaddr);
+ return ret;
}
-static void gen8_ppgtt_insert_entries(struct i915_address_space *vm,
- struct sg_table *pages,
- uint64_t start,
- enum i915_cache_level cache_level,
- u32 unused)
+static void gen8_ppgtt_insert_3lvl(struct i915_address_space *vm,
+ struct sg_table *pages,
+ u64 start,
+ enum i915_cache_level cache_level,
+ u32 unused)
{
struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
- struct sg_page_iter sg_iter;
+ struct sgt_dma iter = {
+ .sg = pages->sgl,
+ .dma = sg_dma_address(iter.sg),
+ .max = iter.dma + iter.sg->length,
+ };
+ struct gen8_insert_pte idx = gen8_insert_pte(start);
- __sg_page_iter_start(&sg_iter, pages->sgl, sg_nents(pages->sgl), 0);
+ gen8_ppgtt_insert_pte_entries(ppgtt, &ppgtt->pdp, &iter, &idx,
+ cache_level);
+}
- if (!USES_FULL_48BIT_PPGTT(vm->i915)) {
- gen8_ppgtt_insert_pte_entries(vm, &ppgtt->pdp, &sg_iter, start,
- cache_level);
- } else {
- struct i915_page_directory_pointer *pdp;
- uint64_t pml4e;
- uint64_t length = (uint64_t)pages->orig_nents << PAGE_SHIFT;
+static void gen8_ppgtt_insert_4lvl(struct i915_address_space *vm,
+ struct sg_table *pages,
+ u64 start,
+ enum i915_cache_level cache_level,
+ u32 unused)
+{
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
+ struct sgt_dma iter = {
+ .sg = pages->sgl,
+ .dma = sg_dma_address(iter.sg),
+ .max = iter.dma + iter.sg->length,
+ };
+ struct i915_page_directory_pointer **pdps = ppgtt->pml4.pdps;
+ struct gen8_insert_pte idx = gen8_insert_pte(start);
- gen8_for_each_pml4e(pdp, &ppgtt->pml4, start, length, pml4e) {
- gen8_ppgtt_insert_pte_entries(vm, pdp, &sg_iter,
- start, cache_level);
- }
- }
+ while (gen8_ppgtt_insert_pte_entries(ppgtt, pdps[idx.pml4e++], &iter,
+ &idx, cache_level))
+ GEM_BUG_ON(idx.pml4e >= GEN8_PML4ES_PER_PML4);
}
-static void gen8_free_page_tables(struct drm_i915_private *dev_priv,
+static void gen8_free_page_tables(struct i915_address_space *vm,
struct i915_page_directory *pd)
{
int i;
if (!px_page(pd))
return;
- for_each_set_bit(i, pd->used_pdes, I915_PDES) {
- if (WARN_ON(!pd->page_table[i]))
- continue;
-
- free_pt(dev_priv, pd->page_table[i]);
- pd->page_table[i] = NULL;
+ for (i = 0; i < I915_PDES; i++) {
+ if (pd->page_table[i] != vm->scratch_pt)
+ free_pt(vm, pd->page_table[i]);
}
}
static int gen8_init_scratch(struct i915_address_space *vm)
{
- struct drm_i915_private *dev_priv = vm->i915;
int ret;
- ret = setup_scratch_page(dev_priv, &vm->scratch_page, I915_GFP_DMA);
+ ret = setup_scratch_page(vm, I915_GFP_DMA);
if (ret)
return ret;
- vm->scratch_pt = alloc_pt(dev_priv);
+ vm->scratch_pt = alloc_pt(vm);
if (IS_ERR(vm->scratch_pt)) {
ret = PTR_ERR(vm->scratch_pt);
goto free_scratch_page;
}
- vm->scratch_pd = alloc_pd(dev_priv);
+ vm->scratch_pd = alloc_pd(vm);
if (IS_ERR(vm->scratch_pd)) {
ret = PTR_ERR(vm->scratch_pd);
goto free_pt;
}
- if (USES_FULL_48BIT_PPGTT(dev_priv)) {
- vm->scratch_pdp = alloc_pdp(dev_priv);
+ if (use_4lvl(vm)) {
+ vm->scratch_pdp = alloc_pdp(vm);
if (IS_ERR(vm->scratch_pdp)) {
ret = PTR_ERR(vm->scratch_pdp);
goto free_pd;
gen8_initialize_pt(vm, vm->scratch_pt);
gen8_initialize_pd(vm, vm->scratch_pd);
- if (USES_FULL_48BIT_PPGTT(dev_priv))
+ if (use_4lvl(vm))
gen8_initialize_pdp(vm, vm->scratch_pdp);
return 0;
free_pd:
- free_pd(dev_priv, vm->scratch_pd);
+ free_pd(vm, vm->scratch_pd);
free_pt:
- free_pt(dev_priv, vm->scratch_pt);
+ free_pt(vm, vm->scratch_pt);
free_scratch_page:
- cleanup_scratch_page(dev_priv, &vm->scratch_page);
+ cleanup_scratch_page(vm);
return ret;
}
static int gen8_ppgtt_notify_vgt(struct i915_hw_ppgtt *ppgtt, bool create)
{
+ struct i915_address_space *vm = &ppgtt->base;
+ struct drm_i915_private *dev_priv = vm->i915;
enum vgt_g2v_type msg;
- struct drm_i915_private *dev_priv = ppgtt->base.i915;
int i;
- if (USES_FULL_48BIT_PPGTT(dev_priv)) {
- u64 daddr = px_dma(&ppgtt->pml4);
+ if (use_4lvl(vm)) {
+ const u64 daddr = px_dma(&ppgtt->pml4);
I915_WRITE(vgtif_reg(pdp[0].lo), lower_32_bits(daddr));
I915_WRITE(vgtif_reg(pdp[0].hi), upper_32_bits(daddr));
msg = (create ? VGT_G2V_PPGTT_L4_PAGE_TABLE_CREATE :
VGT_G2V_PPGTT_L4_PAGE_TABLE_DESTROY);
} else {
- for (i = 0; i < GEN8_LEGACY_PDPES; i++) {
- u64 daddr = i915_page_dir_dma_addr(ppgtt, i);
+ for (i = 0; i < GEN8_3LVL_PDPES; i++) {
+ const u64 daddr = i915_page_dir_dma_addr(ppgtt, i);
I915_WRITE(vgtif_reg(pdp[i].lo), lower_32_bits(daddr));
I915_WRITE(vgtif_reg(pdp[i].hi), upper_32_bits(daddr));
static void gen8_free_scratch(struct i915_address_space *vm)
{
- struct drm_i915_private *dev_priv = vm->i915;
-
- if (USES_FULL_48BIT_PPGTT(dev_priv))
- free_pdp(dev_priv, vm->scratch_pdp);
- free_pd(dev_priv, vm->scratch_pd);
- free_pt(dev_priv, vm->scratch_pt);
- cleanup_scratch_page(dev_priv, &vm->scratch_page);
+ if (use_4lvl(vm))
+ free_pdp(vm, vm->scratch_pdp);
+ free_pd(vm, vm->scratch_pd);
+ free_pt(vm, vm->scratch_pt);
+ cleanup_scratch_page(vm);
}
-static void gen8_ppgtt_cleanup_3lvl(struct drm_i915_private *dev_priv,
+static void gen8_ppgtt_cleanup_3lvl(struct i915_address_space *vm,
struct i915_page_directory_pointer *pdp)
{
+ const unsigned int pdpes = i915_pdpes_per_pdp(vm);
int i;
- for_each_set_bit(i, pdp->used_pdpes, I915_PDPES_PER_PDP(dev_priv)) {
- if (WARN_ON(!pdp->page_directory[i]))
+ for (i = 0; i < pdpes; i++) {
+ if (pdp->page_directory[i] == vm->scratch_pd)
continue;
- gen8_free_page_tables(dev_priv, pdp->page_directory[i]);
- free_pd(dev_priv, pdp->page_directory[i]);
+ gen8_free_page_tables(vm, pdp->page_directory[i]);
+ free_pd(vm, pdp->page_directory[i]);
}
- free_pdp(dev_priv, pdp);
+ free_pdp(vm, pdp);
}
static void gen8_ppgtt_cleanup_4lvl(struct i915_hw_ppgtt *ppgtt)
{
- struct drm_i915_private *dev_priv = ppgtt->base.i915;
int i;
- for_each_set_bit(i, ppgtt->pml4.used_pml4es, GEN8_PML4ES_PER_PML4) {
- if (WARN_ON(!ppgtt->pml4.pdps[i]))
+ for (i = 0; i < GEN8_PML4ES_PER_PML4; i++) {
+ if (ppgtt->pml4.pdps[i] == ppgtt->base.scratch_pdp)
continue;
- gen8_ppgtt_cleanup_3lvl(dev_priv, ppgtt->pml4.pdps[i]);
+ gen8_ppgtt_cleanup_3lvl(&ppgtt->base, ppgtt->pml4.pdps[i]);
}
- cleanup_px(dev_priv, &ppgtt->pml4);
+ cleanup_px(&ppgtt->base, &ppgtt->pml4);
}
static void gen8_ppgtt_cleanup(struct i915_address_space *vm)
if (intel_vgpu_active(dev_priv))
gen8_ppgtt_notify_vgt(ppgtt, false);
- if (!USES_FULL_48BIT_PPGTT(dev_priv))
- gen8_ppgtt_cleanup_3lvl(dev_priv, &ppgtt->pdp);
- else
+ if (use_4lvl(vm))
gen8_ppgtt_cleanup_4lvl(ppgtt);
+ else
+ gen8_ppgtt_cleanup_3lvl(&ppgtt->base, &ppgtt->pdp);
gen8_free_scratch(vm);
}
-/**
- * gen8_ppgtt_alloc_pagetabs() - Allocate page tables for VA range.
- * @vm: Master vm structure.
- * @pd: Page directory for this address range.
- * @start: Starting virtual address to begin allocations.
- * @length: Size of the allocations.
- * @new_pts: Bitmap set by function with new allocations. Likely used by the
- * caller to free on error.
- *
- * Allocate the required number of page tables. Extremely similar to
- * gen8_ppgtt_alloc_page_directories(). The main difference is here we are limited by
- * the page directory boundary (instead of the page directory pointer). That
- * boundary is 1GB virtual. Therefore, unlike gen8_ppgtt_alloc_page_directories(), it is
- * possible, and likely that the caller will need to use multiple calls of this
- * function to achieve the appropriate allocation.
- *
- * Return: 0 if success; negative error code otherwise.
- */
-static int gen8_ppgtt_alloc_pagetabs(struct i915_address_space *vm,
- struct i915_page_directory *pd,
- uint64_t start,
- uint64_t length,
- unsigned long *new_pts)
+static int gen8_ppgtt_alloc_pd(struct i915_address_space *vm,
+ struct i915_page_directory *pd,
+ u64 start, u64 length)
{
- struct drm_i915_private *dev_priv = vm->i915;
struct i915_page_table *pt;
- uint32_t pde;
+ u64 from = start;
+ unsigned int pde;
gen8_for_each_pde(pt, pd, start, length, pde) {
- /* Don't reallocate page tables */
- if (test_bit(pde, pd->used_pdes)) {
- /* Scratch is never allocated this way */
- WARN_ON(pt == vm->scratch_pt);
- continue;
- }
-
- pt = alloc_pt(dev_priv);
- if (IS_ERR(pt))
- goto unwind_out;
-
- gen8_initialize_pt(vm, pt);
- pd->page_table[pde] = pt;
- __set_bit(pde, new_pts);
- trace_i915_page_table_entry_alloc(vm, pde, start, GEN8_PDE_SHIFT);
- }
-
- return 0;
-
-unwind_out:
- for_each_set_bit(pde, new_pts, I915_PDES)
- free_pt(dev_priv, pd->page_table[pde]);
-
- return -ENOMEM;
-}
-
-/**
- * gen8_ppgtt_alloc_page_directories() - Allocate page directories for VA range.
- * @vm: Master vm structure.
- * @pdp: Page directory pointer for this address range.
- * @start: Starting virtual address to begin allocations.
- * @length: Size of the allocations.
- * @new_pds: Bitmap set by function with new allocations. Likely used by the
- * caller to free on error.
- *
- * Allocate the required number of page directories starting at the pde index of
- * @start, and ending at the pde index @start + @length. This function will skip
- * over already allocated page directories within the range, and only allocate
- * new ones, setting the appropriate pointer within the pdp as well as the
- * correct position in the bitmap @new_pds.
- *
- * The function will only allocate the pages within the range for a give page
- * directory pointer. In other words, if @start + @length straddles a virtually
- * addressed PDP boundary (512GB for 4k pages), there will be more allocations
- * required by the caller, This is not currently possible, and the BUG in the
- * code will prevent it.
- *
- * Return: 0 if success; negative error code otherwise.
- */
-static int
-gen8_ppgtt_alloc_page_directories(struct i915_address_space *vm,
- struct i915_page_directory_pointer *pdp,
- uint64_t start,
- uint64_t length,
- unsigned long *new_pds)
-{
- struct drm_i915_private *dev_priv = vm->i915;
- struct i915_page_directory *pd;
- uint32_t pdpe;
- uint32_t pdpes = I915_PDPES_PER_PDP(dev_priv);
-
- WARN_ON(!bitmap_empty(new_pds, pdpes));
-
- gen8_for_each_pdpe(pd, pdp, start, length, pdpe) {
- if (test_bit(pdpe, pdp->used_pdpes))
- continue;
-
- pd = alloc_pd(dev_priv);
- if (IS_ERR(pd))
- goto unwind_out;
-
- gen8_initialize_pd(vm, pd);
- pdp->page_directory[pdpe] = pd;
- __set_bit(pdpe, new_pds);
- trace_i915_page_directory_entry_alloc(vm, pdpe, start, GEN8_PDPE_SHIFT);
- }
+ if (pt == vm->scratch_pt) {
+ pt = alloc_pt(vm);
+ if (IS_ERR(pt))
+ goto unwind;
- return 0;
-
-unwind_out:
- for_each_set_bit(pdpe, new_pds, pdpes)
- free_pd(dev_priv, pdp->page_directory[pdpe]);
-
- return -ENOMEM;
-}
+ gen8_initialize_pt(vm, pt);
-/**
- * gen8_ppgtt_alloc_page_dirpointers() - Allocate pdps for VA range.
- * @vm: Master vm structure.
- * @pml4: Page map level 4 for this address range.
- * @start: Starting virtual address to begin allocations.
- * @length: Size of the allocations.
- * @new_pdps: Bitmap set by function with new allocations. Likely used by the
- * caller to free on error.
- *
- * Allocate the required number of page directory pointers. Extremely similar to
- * gen8_ppgtt_alloc_page_directories() and gen8_ppgtt_alloc_pagetabs().
- * The main difference is here we are limited by the pml4 boundary (instead of
- * the page directory pointer).
- *
- * Return: 0 if success; negative error code otherwise.
- */
-static int
-gen8_ppgtt_alloc_page_dirpointers(struct i915_address_space *vm,
- struct i915_pml4 *pml4,
- uint64_t start,
- uint64_t length,
- unsigned long *new_pdps)
-{
- struct drm_i915_private *dev_priv = vm->i915;
- struct i915_page_directory_pointer *pdp;
- uint32_t pml4e;
-
- WARN_ON(!bitmap_empty(new_pdps, GEN8_PML4ES_PER_PML4));
-
- gen8_for_each_pml4e(pdp, pml4, start, length, pml4e) {
- if (!test_bit(pml4e, pml4->used_pml4es)) {
- pdp = alloc_pdp(dev_priv);
- if (IS_ERR(pdp))
- goto unwind_out;
-
- gen8_initialize_pdp(vm, pdp);
- pml4->pdps[pml4e] = pdp;
- __set_bit(pml4e, new_pdps);
- trace_i915_page_directory_pointer_entry_alloc(vm,
- pml4e,
- start,
- GEN8_PML4E_SHIFT);
+ gen8_ppgtt_set_pde(vm, pd, pt, pde);
+ pd->used_pdes++;
+ GEM_BUG_ON(pd->used_pdes > I915_PDES);
}
- }
-
- return 0;
-
-unwind_out:
- for_each_set_bit(pml4e, new_pdps, GEN8_PML4ES_PER_PML4)
- free_pdp(dev_priv, pml4->pdps[pml4e]);
-
- return -ENOMEM;
-}
-
-static void
-free_gen8_temp_bitmaps(unsigned long *new_pds, unsigned long *new_pts)
-{
- kfree(new_pts);
- kfree(new_pds);
-}
-
-/* Fills in the page directory bitmap, and the array of page tables bitmap. Both
- * of these are based on the number of PDPEs in the system.
- */
-static
-int __must_check alloc_gen8_temp_bitmaps(unsigned long **new_pds,
- unsigned long **new_pts,
- uint32_t pdpes)
-{
- unsigned long *pds;
- unsigned long *pts;
-
- pds = kcalloc(BITS_TO_LONGS(pdpes), sizeof(unsigned long), GFP_TEMPORARY);
- if (!pds)
- return -ENOMEM;
-
- pts = kcalloc(pdpes, BITS_TO_LONGS(I915_PDES) * sizeof(unsigned long),
- GFP_TEMPORARY);
- if (!pts)
- goto err_out;
-
- *new_pds = pds;
- *new_pts = pts;
+ pt->used_ptes += gen8_pte_count(start, length);
+ }
return 0;
-err_out:
- free_gen8_temp_bitmaps(pds, pts);
+unwind:
+ gen8_ppgtt_clear_pd(vm, pd, from, start - from);
return -ENOMEM;
}
-static int gen8_alloc_va_range_3lvl(struct i915_address_space *vm,
- struct i915_page_directory_pointer *pdp,
- uint64_t start,
- uint64_t length)
+static int gen8_ppgtt_alloc_pdp(struct i915_address_space *vm,
+ struct i915_page_directory_pointer *pdp,
+ u64 start, u64 length)
{
- struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
- unsigned long *new_page_dirs, *new_page_tables;
- struct drm_i915_private *dev_priv = vm->i915;
struct i915_page_directory *pd;
- const uint64_t orig_start = start;
- const uint64_t orig_length = length;
- uint32_t pdpe;
- uint32_t pdpes = I915_PDPES_PER_PDP(dev_priv);
+ u64 from = start;
+ unsigned int pdpe;
int ret;
- ret = alloc_gen8_temp_bitmaps(&new_page_dirs, &new_page_tables, pdpes);
- if (ret)
- return ret;
-
- /* Do the allocations first so we can easily bail out */
- ret = gen8_ppgtt_alloc_page_directories(vm, pdp, start, length,
- new_page_dirs);
- if (ret) {
- free_gen8_temp_bitmaps(new_page_dirs, new_page_tables);
- return ret;
- }
-
- /* For every page directory referenced, allocate page tables */
gen8_for_each_pdpe(pd, pdp, start, length, pdpe) {
- ret = gen8_ppgtt_alloc_pagetabs(vm, pd, start, length,
- new_page_tables + pdpe * BITS_TO_LONGS(I915_PDES));
- if (ret)
- goto err_out;
- }
-
- start = orig_start;
- length = orig_length;
-
- /* Allocations have completed successfully, so set the bitmaps, and do
- * the mappings. */
- gen8_for_each_pdpe(pd, pdp, start, length, pdpe) {
- gen8_pde_t *const page_directory = kmap_px(pd);
- struct i915_page_table *pt;
- uint64_t pd_len = length;
- uint64_t pd_start = start;
- uint32_t pde;
+ if (pd == vm->scratch_pd) {
+ pd = alloc_pd(vm);
+ if (IS_ERR(pd))
+ goto unwind;
- /* Every pd should be allocated, we just did that above. */
- WARN_ON(!pd);
+ gen8_initialize_pd(vm, pd);
+ gen8_ppgtt_set_pdpe(vm, pdp, pd, pdpe);
+ pdp->used_pdpes++;
+ GEM_BUG_ON(pdp->used_pdpes > i915_pdpes_per_pdp(vm));
- gen8_for_each_pde(pt, pd, pd_start, pd_len, pde) {
- /* Same reasoning as pd */
- WARN_ON(!pt);
- WARN_ON(!pd_len);
- WARN_ON(!gen8_pte_count(pd_start, pd_len));
-
- /* Set our used ptes within the page table */
- bitmap_set(pt->used_ptes,
- gen8_pte_index(pd_start),
- gen8_pte_count(pd_start, pd_len));
-
- /* Our pde is now pointing to the pagetable, pt */
- __set_bit(pde, pd->used_pdes);
-
- /* Map the PDE to the page table */
- page_directory[pde] = gen8_pde_encode(px_dma(pt),
- I915_CACHE_LLC);
- trace_i915_page_table_entry_map(&ppgtt->base, pde, pt,
- gen8_pte_index(start),
- gen8_pte_count(start, length),
- GEN8_PTES);
-
- /* NB: We haven't yet mapped ptes to pages. At this
- * point we're still relying on insert_entries() */
+ mark_tlbs_dirty(i915_vm_to_ppgtt(vm));
}
- kunmap_px(ppgtt, page_directory);
- __set_bit(pdpe, pdp->used_pdpes);
- gen8_setup_pdpe(ppgtt, pdp, pd, pdpe);
+ ret = gen8_ppgtt_alloc_pd(vm, pd, start, length);
+ if (unlikely(ret))
+ goto unwind_pd;
}
- free_gen8_temp_bitmaps(new_page_dirs, new_page_tables);
- mark_tlbs_dirty(ppgtt);
return 0;
-err_out:
- while (pdpe--) {
- unsigned long temp;
-
- for_each_set_bit(temp, new_page_tables + pdpe *
- BITS_TO_LONGS(I915_PDES), I915_PDES)
- free_pt(dev_priv,
- pdp->page_directory[pdpe]->page_table[temp]);
+unwind_pd:
+ if (!pd->used_pdes) {
+ gen8_ppgtt_set_pdpe(vm, pdp, vm->scratch_pd, pdpe);
+ GEM_BUG_ON(!pdp->used_pdpes);
+ pdp->used_pdpes--;
+ free_pd(vm, pd);
}
+unwind:
+ gen8_ppgtt_clear_pdp(vm, pdp, from, start - from);
+ return -ENOMEM;
+}
- for_each_set_bit(pdpe, new_page_dirs, pdpes)
- free_pd(dev_priv, pdp->page_directory[pdpe]);
-
- free_gen8_temp_bitmaps(new_page_dirs, new_page_tables);
- mark_tlbs_dirty(ppgtt);
- return ret;
+static int gen8_ppgtt_alloc_3lvl(struct i915_address_space *vm,
+ u64 start, u64 length)
+{
+ return gen8_ppgtt_alloc_pdp(vm,
+ &i915_vm_to_ppgtt(vm)->pdp, start, length);
}
-static int gen8_alloc_va_range_4lvl(struct i915_address_space *vm,
- struct i915_pml4 *pml4,
- uint64_t start,
- uint64_t length)
+static int gen8_ppgtt_alloc_4lvl(struct i915_address_space *vm,
+ u64 start, u64 length)
{
- DECLARE_BITMAP(new_pdps, GEN8_PML4ES_PER_PML4);
struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
+ struct i915_pml4 *pml4 = &ppgtt->pml4;
struct i915_page_directory_pointer *pdp;
- uint64_t pml4e;
- int ret = 0;
-
- /* Do the pml4 allocations first, so we don't need to track the newly
- * allocated tables below the pdp */
- bitmap_zero(new_pdps, GEN8_PML4ES_PER_PML4);
-
- /* The pagedirectory and pagetable allocations are done in the shared 3
- * and 4 level code. Just allocate the pdps.
- */
- ret = gen8_ppgtt_alloc_page_dirpointers(vm, pml4, start, length,
- new_pdps);
- if (ret)
- return ret;
-
- WARN(bitmap_weight(new_pdps, GEN8_PML4ES_PER_PML4) > 2,
- "The allocation has spanned more than 512GB. "
- "It is highly likely this is incorrect.");
+ u64 from = start;
+ u32 pml4e;
+ int ret;
gen8_for_each_pml4e(pdp, pml4, start, length, pml4e) {
- WARN_ON(!pdp);
+ if (pml4->pdps[pml4e] == vm->scratch_pdp) {
+ pdp = alloc_pdp(vm);
+ if (IS_ERR(pdp))
+ goto unwind;
- ret = gen8_alloc_va_range_3lvl(vm, pdp, start, length);
- if (ret)
- goto err_out;
+ gen8_initialize_pdp(vm, pdp);
+ gen8_ppgtt_set_pml4e(pml4, pdp, pml4e);
+ }
- gen8_setup_pml4e(ppgtt, pml4, pdp, pml4e);
+ ret = gen8_ppgtt_alloc_pdp(vm, pdp, start, length);
+ if (unlikely(ret))
+ goto unwind_pdp;
}
- bitmap_or(pml4->used_pml4es, new_pdps, pml4->used_pml4es,
- GEN8_PML4ES_PER_PML4);
-
return 0;
-err_out:
- for_each_set_bit(pml4e, new_pdps, GEN8_PML4ES_PER_PML4)
- gen8_ppgtt_cleanup_3lvl(vm->i915, pml4->pdps[pml4e]);
-
- return ret;
-}
-
-static int gen8_alloc_va_range(struct i915_address_space *vm,
- uint64_t start, uint64_t length)
-{
- struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
-
- if (USES_FULL_48BIT_PPGTT(vm->i915))
- return gen8_alloc_va_range_4lvl(vm, &ppgtt->pml4, start, length);
- else
- return gen8_alloc_va_range_3lvl(vm, &ppgtt->pdp, start, length);
+unwind_pdp:
+ if (!pdp->used_pdpes) {
+ gen8_ppgtt_set_pml4e(pml4, vm->scratch_pdp, pml4e);
+ free_pdp(vm, pdp);
+ }
+unwind:
+ gen8_ppgtt_clear_4lvl(vm, from, start - from);
+ return -ENOMEM;
}
-static void gen8_dump_pdp(struct i915_page_directory_pointer *pdp,
- uint64_t start, uint64_t length,
+static void gen8_dump_pdp(struct i915_hw_ppgtt *ppgtt,
+ struct i915_page_directory_pointer *pdp,
+ u64 start, u64 length,
gen8_pte_t scratch_pte,
struct seq_file *m)
{
+ struct i915_address_space *vm = &ppgtt->base;
struct i915_page_directory *pd;
- uint32_t pdpe;
+ u32 pdpe;
gen8_for_each_pdpe(pd, pdp, start, length, pdpe) {
struct i915_page_table *pt;
- uint64_t pd_len = length;
- uint64_t pd_start = start;
- uint32_t pde;
+ u64 pd_len = length;
+ u64 pd_start = start;
+ u32 pde;
- if (!test_bit(pdpe, pdp->used_pdpes))
+ if (pdp->page_directory[pdpe] == ppgtt->base.scratch_pd)
continue;
seq_printf(m, "\tPDPE #%d\n", pdpe);
gen8_for_each_pde(pt, pd, pd_start, pd_len, pde) {
- uint32_t pte;
+ u32 pte;
gen8_pte_t *pt_vaddr;
- if (!test_bit(pde, pd->used_pdes))
+ if (pd->page_table[pde] == ppgtt->base.scratch_pt)
continue;
- pt_vaddr = kmap_px(pt);
+ pt_vaddr = kmap_atomic_px(pt);
for (pte = 0; pte < GEN8_PTES; pte += 4) {
- uint64_t va =
- (pdpe << GEN8_PDPE_SHIFT) |
- (pde << GEN8_PDE_SHIFT) |
- (pte << GEN8_PTE_SHIFT);
+ u64 va = (pdpe << GEN8_PDPE_SHIFT |
+ pde << GEN8_PDE_SHIFT |
+ pte << GEN8_PTE_SHIFT);
int i;
bool found = false;
}
seq_puts(m, "\n");
}
- /* don't use kunmap_px, it could trigger
- * an unnecessary flush.
- */
kunmap_atomic(pt_vaddr);
}
}
static void gen8_dump_ppgtt(struct i915_hw_ppgtt *ppgtt, struct seq_file *m)
{
struct i915_address_space *vm = &ppgtt->base;
- uint64_t start = ppgtt->base.start;
- uint64_t length = ppgtt->base.total;
- gen8_pte_t scratch_pte = gen8_pte_encode(vm->scratch_page.daddr,
- I915_CACHE_LLC);
+ const gen8_pte_t scratch_pte =
+ gen8_pte_encode(vm->scratch_page.daddr, I915_CACHE_LLC);
+ u64 start = 0, length = ppgtt->base.total;
- if (!USES_FULL_48BIT_PPGTT(vm->i915)) {
- gen8_dump_pdp(&ppgtt->pdp, start, length, scratch_pte, m);
- } else {
- uint64_t pml4e;
+ if (use_4lvl(vm)) {
+ u64 pml4e;
struct i915_pml4 *pml4 = &ppgtt->pml4;
struct i915_page_directory_pointer *pdp;
gen8_for_each_pml4e(pdp, pml4, start, length, pml4e) {
- if (!test_bit(pml4e, pml4->used_pml4es))
+ if (pml4->pdps[pml4e] == ppgtt->base.scratch_pdp)
continue;
seq_printf(m, " PML4E #%llu\n", pml4e);
- gen8_dump_pdp(pdp, start, length, scratch_pte, m);
+ gen8_dump_pdp(ppgtt, pdp, start, length, scratch_pte, m);
}
+ } else {
+ gen8_dump_pdp(ppgtt, &ppgtt->pdp, start, length, scratch_pte, m);
}
}
-static int gen8_preallocate_top_level_pdps(struct i915_hw_ppgtt *ppgtt)
+static int gen8_preallocate_top_level_pdp(struct i915_hw_ppgtt *ppgtt)
{
- unsigned long *new_page_dirs, *new_page_tables;
- uint32_t pdpes = I915_PDPES_PER_PDP(to_i915(ppgtt->base.dev));
- int ret;
+ struct i915_address_space *vm = &ppgtt->base;
+ struct i915_page_directory_pointer *pdp = &ppgtt->pdp;
+ struct i915_page_directory *pd;
+ u64 start = 0, length = ppgtt->base.total;
+ u64 from = start;
+ unsigned int pdpe;
- /* We allocate temp bitmap for page tables for no gain
- * but as this is for init only, lets keep the things simple
- */
- ret = alloc_gen8_temp_bitmaps(&new_page_dirs, &new_page_tables, pdpes);
- if (ret)
- return ret;
+ gen8_for_each_pdpe(pd, pdp, start, length, pdpe) {
+ pd = alloc_pd(vm);
+ if (IS_ERR(pd))
+ goto unwind;
- /* Allocate for all pdps regardless of how the ppgtt
- * was defined.
- */
- ret = gen8_ppgtt_alloc_page_directories(&ppgtt->base, &ppgtt->pdp,
- 0, 1ULL << 32,
- new_page_dirs);
- if (!ret)
- *ppgtt->pdp.used_pdpes = *new_page_dirs;
+ gen8_initialize_pd(vm, pd);
+ gen8_ppgtt_set_pdpe(vm, pdp, pd, pdpe);
+ pdp->used_pdpes++;
+ }
- free_gen8_temp_bitmaps(new_page_dirs, new_page_tables);
+ pdp->used_pdpes++; /* never remove */
+ return 0;
- return ret;
+unwind:
+ start -= from;
+ gen8_for_each_pdpe(pd, pdp, from, start, pdpe) {
+ gen8_ppgtt_set_pdpe(vm, pdp, vm->scratch_pd, pdpe);
+ free_pd(vm, pd);
+ }
+ pdp->used_pdpes = 0;
+ return -ENOMEM;
}
/*
*/
static int gen8_ppgtt_init(struct i915_hw_ppgtt *ppgtt)
{
- struct drm_i915_private *dev_priv = ppgtt->base.i915;
+ struct i915_address_space *vm = &ppgtt->base;
+ struct drm_i915_private *dev_priv = vm->i915;
int ret;
+ ppgtt->base.total = USES_FULL_48BIT_PPGTT(dev_priv) ?
+ 1ULL << 48 :
+ 1ULL << 32;
+
ret = gen8_init_scratch(&ppgtt->base);
- if (ret)
+ if (ret) {
+ ppgtt->base.total = 0;
return ret;
+ }
- ppgtt->base.start = 0;
- ppgtt->base.cleanup = gen8_ppgtt_cleanup;
- ppgtt->base.allocate_va_range = gen8_alloc_va_range;
- ppgtt->base.insert_entries = gen8_ppgtt_insert_entries;
- ppgtt->base.clear_range = gen8_ppgtt_clear_range;
- ppgtt->base.unbind_vma = ppgtt_unbind_vma;
- ppgtt->base.bind_vma = ppgtt_bind_vma;
- ppgtt->debug_dump = gen8_dump_ppgtt;
+ /* There are only few exceptions for gen >=6. chv and bxt.
+ * And we are not sure about the latter so play safe for now.
+ */
+ if (IS_CHERRYVIEW(dev_priv) || IS_BROXTON(dev_priv))
+ ppgtt->base.pt_kmap_wc = true;
- if (USES_FULL_48BIT_PPGTT(dev_priv)) {
- ret = setup_px(dev_priv, &ppgtt->pml4);
+ if (use_4lvl(vm)) {
+ ret = setup_px(&ppgtt->base, &ppgtt->pml4);
if (ret)
goto free_scratch;
gen8_initialize_pml4(&ppgtt->base, &ppgtt->pml4);
- ppgtt->base.total = 1ULL << 48;
- ppgtt->switch_mm = gen8_48b_mm_switch;
+ ppgtt->switch_mm = gen8_mm_switch_4lvl;
+ ppgtt->base.allocate_va_range = gen8_ppgtt_alloc_4lvl;
+ ppgtt->base.insert_entries = gen8_ppgtt_insert_4lvl;
+ ppgtt->base.clear_range = gen8_ppgtt_clear_4lvl;
} else {
- ret = __pdp_init(dev_priv, &ppgtt->pdp);
+ ret = __pdp_init(&ppgtt->base, &ppgtt->pdp);
if (ret)
goto free_scratch;
- ppgtt->base.total = 1ULL << 32;
- ppgtt->switch_mm = gen8_legacy_mm_switch;
- trace_i915_page_directory_pointer_entry_alloc(&ppgtt->base,
- 0, 0,
- GEN8_PML4E_SHIFT);
-
if (intel_vgpu_active(dev_priv)) {
- ret = gen8_preallocate_top_level_pdps(ppgtt);
- if (ret)
+ ret = gen8_preallocate_top_level_pdp(ppgtt);
+ if (ret) {
+ __pdp_fini(&ppgtt->pdp);
goto free_scratch;
+ }
}
+
+ ppgtt->switch_mm = gen8_mm_switch_3lvl;
+ ppgtt->base.allocate_va_range = gen8_ppgtt_alloc_3lvl;
+ ppgtt->base.insert_entries = gen8_ppgtt_insert_3lvl;
+ ppgtt->base.clear_range = gen8_ppgtt_clear_3lvl;
}
if (intel_vgpu_active(dev_priv))
gen8_ppgtt_notify_vgt(ppgtt, true);
+ ppgtt->base.cleanup = gen8_ppgtt_cleanup;
+ ppgtt->base.unbind_vma = ppgtt_unbind_vma;
+ ppgtt->base.bind_vma = ppgtt_bind_vma;
+ ppgtt->debug_dump = gen8_dump_ppgtt;
+
return 0;
free_scratch:
struct i915_address_space *vm = &ppgtt->base;
struct i915_page_table *unused;
gen6_pte_t scratch_pte;
- uint32_t pd_entry;
- uint32_t pte, pde;
- uint32_t start = ppgtt->base.start, length = ppgtt->base.total;
+ u32 pd_entry, pte, pde;
+ u32 start = 0, length = ppgtt->base.total;
scratch_pte = vm->pte_encode(vm->scratch_page.daddr,
I915_CACHE_LLC, 0);
expected);
seq_printf(m, "\tPDE: %x\n", pd_entry);
- pt_vaddr = kmap_px(ppgtt->pd.page_table[pde]);
+ pt_vaddr = kmap_atomic_px(ppgtt->pd.page_table[pde]);
for (pte = 0; pte < GEN6_PTES; pte+=4) {
unsigned long va =
}
seq_puts(m, "\n");
}
- kunmap_px(ppgtt, pt_vaddr);
+ kunmap_atomic(pt_vaddr);
}
}
/* Write pde (index) from the page directory @pd to the page table @pt */
-static void gen6_write_pde(struct i915_page_directory *pd,
- const int pde, struct i915_page_table *pt)
+static inline void gen6_write_pde(const struct i915_hw_ppgtt *ppgtt,
+ const unsigned int pde,
+ const struct i915_page_table *pt)
{
/* Caller needs to make sure the write completes if necessary */
- struct i915_hw_ppgtt *ppgtt =
- container_of(pd, struct i915_hw_ppgtt, pd);
- u32 pd_entry;
-
- pd_entry = GEN6_PDE_ADDR_ENCODE(px_dma(pt));
- pd_entry |= GEN6_PDE_VALID;
-
- writel(pd_entry, ppgtt->pd_addr + pde);
+ writel_relaxed(GEN6_PDE_ADDR_ENCODE(px_dma(pt)) | GEN6_PDE_VALID,
+ ppgtt->pd_addr + pde);
}
/* Write all the page tables found in the ppgtt structure to incrementing page
* directories. */
-static void gen6_write_page_range(struct drm_i915_private *dev_priv,
- struct i915_page_directory *pd,
- uint32_t start, uint32_t length)
+static void gen6_write_page_range(struct i915_hw_ppgtt *ppgtt,
+ u32 start, u32 length)
{
- struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct i915_page_table *pt;
- uint32_t pde;
+ unsigned int pde;
- gen6_for_each_pde(pt, pd, start, length, pde)
- gen6_write_pde(pd, pde, pt);
+ gen6_for_each_pde(pt, &ppgtt->pd, start, length, pde)
+ gen6_write_pde(ppgtt, pde, pt);
- /* Make sure write is complete before other code can use this page
- * table. Also require for WC mapped PTEs */
- readl(ggtt->gsm);
+ mark_tlbs_dirty(ppgtt);
+ wmb();
}
-static uint32_t get_pd_offset(struct i915_hw_ppgtt *ppgtt)
+static inline u32 get_pd_offset(struct i915_hw_ppgtt *ppgtt)
{
- BUG_ON(ppgtt->pd.base.ggtt_offset & 0x3f);
-
- return (ppgtt->pd.base.ggtt_offset / 64) << 16;
+ GEM_BUG_ON(ppgtt->pd.base.ggtt_offset & 0x3f);
+ return ppgtt->pd.base.ggtt_offset << 10;
}
static int hsw_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_ring *ring = req->ring;
struct intel_engine_cs *engine = req->engine;
- int ret;
+ u32 *cs;
/* NB: TLBs must be flushed and invalidated before a switch */
- ret = engine->emit_flush(req, EMIT_INVALIDATE | EMIT_FLUSH);
- if (ret)
- return ret;
-
- ret = intel_ring_begin(req, 6);
- if (ret)
- return ret;
-
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(2));
- intel_ring_emit_reg(ring, RING_PP_DIR_DCLV(engine));
- intel_ring_emit(ring, PP_DIR_DCLV_2G);
- intel_ring_emit_reg(ring, RING_PP_DIR_BASE(engine));
- intel_ring_emit(ring, get_pd_offset(ppgtt));
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ cs = intel_ring_begin(req, 6);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
+
+ *cs++ = MI_LOAD_REGISTER_IMM(2);
+ *cs++ = i915_mmio_reg_offset(RING_PP_DIR_DCLV(engine));
+ *cs++ = PP_DIR_DCLV_2G;
+ *cs++ = i915_mmio_reg_offset(RING_PP_DIR_BASE(engine));
+ *cs++ = get_pd_offset(ppgtt);
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
static int gen7_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_ring *ring = req->ring;
struct intel_engine_cs *engine = req->engine;
- int ret;
+ u32 *cs;
/* NB: TLBs must be flushed and invalidated before a switch */
- ret = engine->emit_flush(req, EMIT_INVALIDATE | EMIT_FLUSH);
- if (ret)
- return ret;
-
- ret = intel_ring_begin(req, 6);
- if (ret)
- return ret;
-
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(2));
- intel_ring_emit_reg(ring, RING_PP_DIR_DCLV(engine));
- intel_ring_emit(ring, PP_DIR_DCLV_2G);
- intel_ring_emit_reg(ring, RING_PP_DIR_BASE(engine));
- intel_ring_emit(ring, get_pd_offset(ppgtt));
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
-
- /* XXX: RCS is the only one to auto invalidate the TLBs? */
- if (engine->id != RCS) {
- ret = engine->emit_flush(req, EMIT_INVALIDATE | EMIT_FLUSH);
- if (ret)
- return ret;
- }
+ cs = intel_ring_begin(req, 6);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
+
+ *cs++ = MI_LOAD_REGISTER_IMM(2);
+ *cs++ = i915_mmio_reg_offset(RING_PP_DIR_DCLV(engine));
+ *cs++ = PP_DIR_DCLV_2G;
+ *cs++ = i915_mmio_reg_offset(RING_PP_DIR_BASE(engine));
+ *cs++ = get_pd_offset(ppgtt);
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
static void gen7_ppgtt_enable(struct drm_i915_private *dev_priv)
{
struct intel_engine_cs *engine;
- uint32_t ecochk, ecobits;
+ u32 ecochk, ecobits;
enum intel_engine_id id;
ecobits = I915_READ(GAC_ECO_BITS);
static void gen6_ppgtt_enable(struct drm_i915_private *dev_priv)
{
- uint32_t ecochk, gab_ctl, ecobits;
+ u32 ecochk, gab_ctl, ecobits;
ecobits = I915_READ(GAC_ECO_BITS);
I915_WRITE(GAC_ECO_BITS, ecobits | ECOBITS_SNB_BIT |
/* PPGTT support for Sandybdrige/Gen6 and later */
static void gen6_ppgtt_clear_range(struct i915_address_space *vm,
- uint64_t start,
- uint64_t length)
+ u64 start, u64 length)
{
struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
- gen6_pte_t *pt_vaddr, scratch_pte;
- unsigned first_entry = start >> PAGE_SHIFT;
- unsigned num_entries = length >> PAGE_SHIFT;
- unsigned act_pt = first_entry / GEN6_PTES;
- unsigned first_pte = first_entry % GEN6_PTES;
- unsigned last_pte, i;
-
- scratch_pte = vm->pte_encode(vm->scratch_page.daddr,
- I915_CACHE_LLC, 0);
+ unsigned int first_entry = start >> PAGE_SHIFT;
+ unsigned int pde = first_entry / GEN6_PTES;
+ unsigned int pte = first_entry % GEN6_PTES;
+ unsigned int num_entries = length >> PAGE_SHIFT;
+ gen6_pte_t scratch_pte =
+ vm->pte_encode(vm->scratch_page.daddr, I915_CACHE_LLC, 0);
while (num_entries) {
- last_pte = first_pte + num_entries;
- if (last_pte > GEN6_PTES)
- last_pte = GEN6_PTES;
+ struct i915_page_table *pt = ppgtt->pd.page_table[pde++];
+ unsigned int end = min(pte + num_entries, GEN6_PTES);
+ gen6_pte_t *vaddr;
- pt_vaddr = kmap_px(ppgtt->pd.page_table[act_pt]);
+ num_entries -= end - pte;
- for (i = first_pte; i < last_pte; i++)
- pt_vaddr[i] = scratch_pte;
+ /* Note that the hw doesn't support removing PDE on the fly
+ * (they are cached inside the context with no means to
+ * invalidate the cache), so we can only reset the PTE
+ * entries back to scratch.
+ */
- kunmap_px(ppgtt, pt_vaddr);
+ vaddr = kmap_atomic_px(pt);
+ do {
+ vaddr[pte++] = scratch_pte;
+ } while (pte < end);
+ kunmap_atomic(vaddr);
- num_entries -= last_pte - first_pte;
- first_pte = 0;
- act_pt++;
+ pte = 0;
}
}
static void gen6_ppgtt_insert_entries(struct i915_address_space *vm,
struct sg_table *pages,
- uint64_t start,
- enum i915_cache_level cache_level, u32 flags)
+ u64 start,
+ enum i915_cache_level cache_level,
+ u32 flags)
{
struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
unsigned first_entry = start >> PAGE_SHIFT;
unsigned act_pt = first_entry / GEN6_PTES;
unsigned act_pte = first_entry % GEN6_PTES;
- gen6_pte_t *pt_vaddr = NULL;
- struct sgt_iter sgt_iter;
- dma_addr_t addr;
+ const u32 pte_encode = vm->pte_encode(0, cache_level, flags);
+ struct sgt_dma iter;
+ gen6_pte_t *vaddr;
+
+ vaddr = kmap_atomic_px(ppgtt->pd.page_table[act_pt]);
+ iter.sg = pages->sgl;
+ iter.dma = sg_dma_address(iter.sg);
+ iter.max = iter.dma + iter.sg->length;
+ do {
+ vaddr[act_pte] = pte_encode | GEN6_PTE_ADDR_ENCODE(iter.dma);
- for_each_sgt_dma(addr, sgt_iter, pages) {
- if (pt_vaddr == NULL)
- pt_vaddr = kmap_px(ppgtt->pd.page_table[act_pt]);
+ iter.dma += PAGE_SIZE;
+ if (iter.dma == iter.max) {
+ iter.sg = __sg_next(iter.sg);
+ if (!iter.sg)
+ break;
- pt_vaddr[act_pte] =
- vm->pte_encode(addr, cache_level, flags);
+ iter.dma = sg_dma_address(iter.sg);
+ iter.max = iter.dma + iter.sg->length;
+ }
if (++act_pte == GEN6_PTES) {
- kunmap_px(ppgtt, pt_vaddr);
- pt_vaddr = NULL;
- act_pt++;
+ kunmap_atomic(vaddr);
+ vaddr = kmap_atomic_px(ppgtt->pd.page_table[++act_pt]);
act_pte = 0;
}
- }
-
- if (pt_vaddr)
- kunmap_px(ppgtt, pt_vaddr);
+ } while (1);
+ kunmap_atomic(vaddr);
}
static int gen6_alloc_va_range(struct i915_address_space *vm,
- uint64_t start_in, uint64_t length_in)
+ u64 start, u64 length)
{
- DECLARE_BITMAP(new_page_tables, I915_PDES);
- struct drm_i915_private *dev_priv = vm->i915;
- struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
struct i915_page_table *pt;
- uint32_t start, length, start_save, length_save;
- uint32_t pde;
- int ret;
-
- start = start_save = start_in;
- length = length_save = length_in;
+ u64 from = start;
+ unsigned int pde;
+ bool flush = false;
- bitmap_zero(new_page_tables, I915_PDES);
-
- /* The allocation is done in two stages so that we can bail out with
- * minimal amount of pain. The first stage finds new page tables that
- * need allocation. The second stage marks use ptes within the page
- * tables.
- */
gen6_for_each_pde(pt, &ppgtt->pd, start, length, pde) {
- if (pt != vm->scratch_pt) {
- WARN_ON(bitmap_empty(pt->used_ptes, GEN6_PTES));
- continue;
- }
-
- /* We've already allocated a page table */
- WARN_ON(!bitmap_empty(pt->used_ptes, GEN6_PTES));
+ if (pt == vm->scratch_pt) {
+ pt = alloc_pt(vm);
+ if (IS_ERR(pt))
+ goto unwind_out;
- pt = alloc_pt(dev_priv);
- if (IS_ERR(pt)) {
- ret = PTR_ERR(pt);
- goto unwind_out;
+ gen6_initialize_pt(vm, pt);
+ ppgtt->pd.page_table[pde] = pt;
+ gen6_write_pde(ppgtt, pde, pt);
+ flush = true;
}
-
- gen6_initialize_pt(vm, pt);
-
- ppgtt->pd.page_table[pde] = pt;
- __set_bit(pde, new_page_tables);
- trace_i915_page_table_entry_alloc(vm, pde, start, GEN6_PDE_SHIFT);
}
- start = start_save;
- length = length_save;
-
- gen6_for_each_pde(pt, &ppgtt->pd, start, length, pde) {
- DECLARE_BITMAP(tmp_bitmap, GEN6_PTES);
-
- bitmap_zero(tmp_bitmap, GEN6_PTES);
- bitmap_set(tmp_bitmap, gen6_pte_index(start),
- gen6_pte_count(start, length));
-
- if (__test_and_clear_bit(pde, new_page_tables))
- gen6_write_pde(&ppgtt->pd, pde, pt);
-
- trace_i915_page_table_entry_map(vm, pde, pt,
- gen6_pte_index(start),
- gen6_pte_count(start, length),
- GEN6_PTES);
- bitmap_or(pt->used_ptes, tmp_bitmap, pt->used_ptes,
- GEN6_PTES);
+ if (flush) {
+ mark_tlbs_dirty(ppgtt);
+ wmb();
}
- WARN_ON(!bitmap_empty(new_page_tables, I915_PDES));
-
- /* Make sure write is complete before other code can use this page
- * table. Also require for WC mapped PTEs */
- readl(ggtt->gsm);
-
- mark_tlbs_dirty(ppgtt);
return 0;
unwind_out:
- for_each_set_bit(pde, new_page_tables, I915_PDES) {
- struct i915_page_table *pt = ppgtt->pd.page_table[pde];
-
- ppgtt->pd.page_table[pde] = vm->scratch_pt;
- free_pt(dev_priv, pt);
- }
-
- mark_tlbs_dirty(ppgtt);
- return ret;
+ gen6_ppgtt_clear_range(vm, from, start);
+ return -ENOMEM;
}
static int gen6_init_scratch(struct i915_address_space *vm)
{
- struct drm_i915_private *dev_priv = vm->i915;
int ret;
- ret = setup_scratch_page(dev_priv, &vm->scratch_page, I915_GFP_DMA);
+ ret = setup_scratch_page(vm, I915_GFP_DMA);
if (ret)
return ret;
- vm->scratch_pt = alloc_pt(dev_priv);
+ vm->scratch_pt = alloc_pt(vm);
if (IS_ERR(vm->scratch_pt)) {
- cleanup_scratch_page(dev_priv, &vm->scratch_page);
+ cleanup_scratch_page(vm);
return PTR_ERR(vm->scratch_pt);
}
static void gen6_free_scratch(struct i915_address_space *vm)
{
- struct drm_i915_private *dev_priv = vm->i915;
-
- free_pt(dev_priv, vm->scratch_pt);
- cleanup_scratch_page(dev_priv, &vm->scratch_page);
+ free_pt(vm, vm->scratch_pt);
+ cleanup_scratch_page(vm);
}
static void gen6_ppgtt_cleanup(struct i915_address_space *vm)
{
struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
struct i915_page_directory *pd = &ppgtt->pd;
- struct drm_i915_private *dev_priv = vm->i915;
struct i915_page_table *pt;
- uint32_t pde;
+ u32 pde;
drm_mm_remove_node(&ppgtt->node);
gen6_for_all_pdes(pt, pd, pde)
if (pt != vm->scratch_pt)
- free_pt(dev_priv, pt);
+ free_pt(vm, pt);
gen6_free_scratch(vm);
}
if (ppgtt->node.start < ggtt->mappable_end)
DRM_DEBUG("Forced to use aperture for PDEs\n");
+ ppgtt->pd.base.ggtt_offset =
+ ppgtt->node.start / PAGE_SIZE * sizeof(gen6_pte_t);
+
+ ppgtt->pd_addr = (gen6_pte_t __iomem *)ggtt->gsm +
+ ppgtt->pd.base.ggtt_offset / sizeof(gen6_pte_t);
+
return 0;
err_out:
}
static void gen6_scratch_va_range(struct i915_hw_ppgtt *ppgtt,
- uint64_t start, uint64_t length)
+ u64 start, u64 length)
{
struct i915_page_table *unused;
- uint32_t pde;
+ u32 pde;
gen6_for_each_pde(unused, &ppgtt->pd, start, length, pde)
ppgtt->pd.page_table[pde] = ppgtt->base.scratch_pt;
if (ret)
return ret;
- ppgtt->base.allocate_va_range = gen6_alloc_va_range;
+ ppgtt->base.total = I915_PDES * GEN6_PTES * PAGE_SIZE;
+
+ gen6_scratch_va_range(ppgtt, 0, ppgtt->base.total);
+ gen6_write_page_range(ppgtt, 0, ppgtt->base.total);
+
+ ret = gen6_alloc_va_range(&ppgtt->base, 0, ppgtt->base.total);
+ if (ret) {
+ gen6_ppgtt_cleanup(&ppgtt->base);
+ return ret;
+ }
+
ppgtt->base.clear_range = gen6_ppgtt_clear_range;
ppgtt->base.insert_entries = gen6_ppgtt_insert_entries;
ppgtt->base.unbind_vma = ppgtt_unbind_vma;
ppgtt->base.bind_vma = ppgtt_bind_vma;
ppgtt->base.cleanup = gen6_ppgtt_cleanup;
- ppgtt->base.start = 0;
- ppgtt->base.total = I915_PDES * GEN6_PTES * PAGE_SIZE;
ppgtt->debug_dump = gen6_dump_ppgtt;
- ppgtt->pd.base.ggtt_offset =
- ppgtt->node.start / PAGE_SIZE * sizeof(gen6_pte_t);
-
- ppgtt->pd_addr = (gen6_pte_t __iomem *)ggtt->gsm +
- ppgtt->pd.base.ggtt_offset / sizeof(gen6_pte_t);
-
- gen6_scratch_va_range(ppgtt, 0, ppgtt->base.total);
-
- gen6_write_page_range(dev_priv, &ppgtt->pd, 0, ppgtt->base.total);
-
DRM_DEBUG_DRIVER("Allocated pde space (%lldM) at GTT entry: %llx\n",
ppgtt->node.size >> 20,
ppgtt->node.start / PAGE_SIZE);
- DRM_DEBUG("Adding PPGTT at offset %x\n",
- ppgtt->pd.base.ggtt_offset << 10);
+ DRM_DEBUG_DRIVER("Adding PPGTT at offset %x\n",
+ ppgtt->pd.base.ggtt_offset << 10);
return 0;
}
struct drm_i915_private *dev_priv)
{
ppgtt->base.i915 = dev_priv;
+ ppgtt->base.dma = &dev_priv->drm.pdev->dev;
if (INTEL_INFO(dev_priv)->gen < 8)
return gen6_ppgtt_init(ppgtt);
const char *name)
{
i915_gem_timeline_init(dev_priv, &vm->timeline, name);
- drm_mm_init(&vm->mm, vm->start, vm->total);
+
+ drm_mm_init(&vm->mm, 0, vm->total);
+ vm->mm.head_node.color = I915_COLOR_UNEVICTABLE;
+
INIT_LIST_HEAD(&vm->active_list);
INIT_LIST_HEAD(&vm->inactive_list);
INIT_LIST_HEAD(&vm->unbound_list);
+
list_add_tail(&vm->global_link, &dev_priv->vm_list);
+ pagevec_init(&vm->free_pages, false);
}
static void i915_address_space_fini(struct i915_address_space *vm)
{
+ if (pagevec_count(&vm->free_pages))
+ vm_free_pages_release(vm);
+
i915_gem_timeline_fini(&vm->timeline);
drm_mm_takedown(&vm->mm);
list_del(&vm->global_link);
* called on driver load and after a GPU reset, so you can place
* workarounds here even if they get overwritten by GPU reset.
*/
- /* WaIncreaseDefaultTLBEntries:chv,bdw,skl,bxt */
+ /* WaIncreaseDefaultTLBEntries:chv,bdw,skl,bxt,kbl,glk */
if (IS_BROADWELL(dev_priv))
I915_WRITE(GEN8_L3_LRA_1_GPGPU, GEN8_L3_LRA_1_GPGPU_DEFAULT_VALUE_BDW);
else if (IS_CHERRYVIEW(dev_priv))
I915_WRITE(GEN8_L3_LRA_1_GPGPU, GEN8_L3_LRA_1_GPGPU_DEFAULT_VALUE_CHV);
- else if (IS_SKYLAKE(dev_priv))
+ else if (IS_GEN9_BC(dev_priv))
I915_WRITE(GEN8_L3_LRA_1_GPGPU, GEN9_L3_LRA_1_GPGPU_DEFAULT_VALUE_SKL);
- else if (IS_BROXTON(dev_priv))
+ else if (IS_GEN9_LP(dev_priv))
I915_WRITE(GEN8_L3_LRA_1_GPGPU, GEN9_L3_LRA_1_GPGPU_DEFAULT_VALUE_BXT);
}
-static int i915_ppgtt_init(struct i915_hw_ppgtt *ppgtt,
- struct drm_i915_private *dev_priv,
- struct drm_i915_file_private *file_priv,
- const char *name)
-{
- int ret;
-
- ret = __hw_ppgtt_init(ppgtt, dev_priv);
- if (ret == 0) {
- kref_init(&ppgtt->ref);
- i915_address_space_init(&ppgtt->base, dev_priv, name);
- ppgtt->base.file = file_priv;
- }
-
- return ret;
-}
-
int i915_ppgtt_init_hw(struct drm_i915_private *dev_priv)
{
gtt_write_workarounds(dev_priv);
if (!ppgtt)
return ERR_PTR(-ENOMEM);
- ret = i915_ppgtt_init(ppgtt, dev_priv, fpriv, name);
+ ret = __hw_ppgtt_init(ppgtt, dev_priv);
if (ret) {
kfree(ppgtt);
return ERR_PTR(ret);
}
+ kref_init(&ppgtt->ref);
+ i915_address_space_init(&ppgtt->base, dev_priv, name);
+ ppgtt->base.file = fpriv;
+
trace_i915_ppgtt_create(&ppgtt->base);
return ppgtt;
WARN_ON(!list_empty(&ppgtt->base.inactive_list));
WARN_ON(!list_empty(&ppgtt->base.unbound_list));
- i915_address_space_fini(&ppgtt->base);
-
ppgtt->base.cleanup(&ppgtt->base);
+ i915_address_space_fini(&ppgtt->base);
kfree(ppgtt);
}
i915_check_and_clear_faults(dev_priv);
- ggtt->base.clear_range(&ggtt->base, ggtt->base.start, ggtt->base.total);
+ ggtt->base.clear_range(&ggtt->base, 0, ggtt->base.total);
i915_ggtt_invalidate(dev_priv);
}
static void gen8_ggtt_insert_page(struct i915_address_space *vm,
dma_addr_t addr,
- uint64_t offset,
+ u64 offset,
enum i915_cache_level level,
u32 unused)
{
static void gen8_ggtt_insert_entries(struct i915_address_space *vm,
struct sg_table *st,
- uint64_t start,
- enum i915_cache_level level, u32 unused)
+ u64 start,
+ enum i915_cache_level level,
+ u32 unused)
{
struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm);
struct sgt_iter sgt_iter;
gen8_pte_t __iomem *gtt_entries;
- gen8_pte_t gtt_entry;
+ const gen8_pte_t pte_encode = gen8_pte_encode(0, level);
dma_addr_t addr;
- int i = 0;
- gtt_entries = (gen8_pte_t __iomem *)ggtt->gsm + (start >> PAGE_SHIFT);
-
- for_each_sgt_dma(addr, sgt_iter, st) {
- gtt_entry = gen8_pte_encode(addr, level);
- gen8_set_pte(>t_entries[i++], gtt_entry);
- }
+ gtt_entries = (gen8_pte_t __iomem *)ggtt->gsm;
+ gtt_entries += start >> PAGE_SHIFT;
+ for_each_sgt_dma(addr, sgt_iter, st)
+ gen8_set_pte(gtt_entries++, pte_encode | addr);
- /*
- * XXX: This serves as a posting read to make sure that the PTE has
- * actually been updated. There is some concern that even though
- * registers and PTEs are within the same BAR that they are potentially
- * of NUMA access patterns. Therefore, even with the way we assume
- * hardware should work, we must keep this posting read for paranoia.
- */
- if (i != 0)
- WARN_ON(readq(>t_entries[i-1]) != gtt_entry);
+ wmb();
/* This next bit makes the above posting read even more important. We
* want to flush the TLBs only after we're certain all the PTE updates
ggtt->invalidate(vm->i915);
}
-struct insert_entries {
- struct i915_address_space *vm;
- struct sg_table *st;
- uint64_t start;
- enum i915_cache_level level;
- u32 flags;
-};
-
-static int gen8_ggtt_insert_entries__cb(void *_arg)
-{
- struct insert_entries *arg = _arg;
- gen8_ggtt_insert_entries(arg->vm, arg->st,
- arg->start, arg->level, arg->flags);
- return 0;
-}
-
-static void gen8_ggtt_insert_entries__BKL(struct i915_address_space *vm,
- struct sg_table *st,
- uint64_t start,
- enum i915_cache_level level,
- u32 flags)
-{
- struct insert_entries arg = { vm, st, start, level, flags };
- stop_machine(gen8_ggtt_insert_entries__cb, &arg, NULL);
-}
-
static void gen6_ggtt_insert_page(struct i915_address_space *vm,
dma_addr_t addr,
- uint64_t offset,
+ u64 offset,
enum i915_cache_level level,
u32 flags)
{
*/
static void gen6_ggtt_insert_entries(struct i915_address_space *vm,
struct sg_table *st,
- uint64_t start,
- enum i915_cache_level level, u32 flags)
+ u64 start,
+ enum i915_cache_level level,
+ u32 flags)
{
struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm);
- struct sgt_iter sgt_iter;
- gen6_pte_t __iomem *gtt_entries;
- gen6_pte_t gtt_entry;
+ gen6_pte_t __iomem *entries = (gen6_pte_t __iomem *)ggtt->gsm;
+ unsigned int i = start >> PAGE_SHIFT;
+ struct sgt_iter iter;
dma_addr_t addr;
- int i = 0;
-
- gtt_entries = (gen6_pte_t __iomem *)ggtt->gsm + (start >> PAGE_SHIFT);
-
- for_each_sgt_dma(addr, sgt_iter, st) {
- gtt_entry = vm->pte_encode(addr, level, flags);
- iowrite32(gtt_entry, >t_entries[i++]);
- }
-
- /* XXX: This serves as a posting read to make sure that the PTE has
- * actually been updated. There is some concern that even though
- * registers and PTEs are within the same BAR that they are potentially
- * of NUMA access patterns. Therefore, even with the way we assume
- * hardware should work, we must keep this posting read for paranoia.
- */
- if (i != 0)
- WARN_ON(readl(>t_entries[i-1]) != gtt_entry);
+ for_each_sgt_dma(addr, iter, st)
+ iowrite32(vm->pte_encode(addr, level, flags), &entries[i++]);
+ wmb();
/* This next bit makes the above posting read even more important. We
* want to flush the TLBs only after we're certain all the PTE updates
}
static void nop_clear_range(struct i915_address_space *vm,
- uint64_t start, uint64_t length)
+ u64 start, u64 length)
{
}
static void gen8_ggtt_clear_range(struct i915_address_space *vm,
- uint64_t start, uint64_t length)
+ u64 start, u64 length)
{
struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm);
unsigned first_entry = start >> PAGE_SHIFT;
unsigned num_entries = length >> PAGE_SHIFT;
- gen8_pte_t scratch_pte, __iomem *gtt_base =
+ const gen8_pte_t scratch_pte =
+ gen8_pte_encode(vm->scratch_page.daddr, I915_CACHE_LLC);
+ gen8_pte_t __iomem *gtt_base =
(gen8_pte_t __iomem *)ggtt->gsm + first_entry;
const int max_entries = ggtt_total_entries(ggtt) - first_entry;
int i;
first_entry, num_entries, max_entries))
num_entries = max_entries;
- scratch_pte = gen8_pte_encode(vm->scratch_page.daddr,
- I915_CACHE_LLC);
for (i = 0; i < num_entries; i++)
gen8_set_pte(>t_base[i], scratch_pte);
- readl(gtt_base);
}
static void gen6_ggtt_clear_range(struct i915_address_space *vm,
- uint64_t start,
- uint64_t length)
+ u64 start, u64 length)
{
struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm);
unsigned first_entry = start >> PAGE_SHIFT;
for (i = 0; i < num_entries; i++)
iowrite32(scratch_pte, >t_base[i]);
- readl(gtt_base);
}
static void i915_ggtt_insert_page(struct i915_address_space *vm,
dma_addr_t addr,
- uint64_t offset,
+ u64 offset,
enum i915_cache_level cache_level,
u32 unused)
{
static void i915_ggtt_insert_entries(struct i915_address_space *vm,
struct sg_table *pages,
- uint64_t start,
- enum i915_cache_level cache_level, u32 unused)
+ u64 start,
+ enum i915_cache_level cache_level,
+ u32 unused)
{
unsigned int flags = (cache_level == I915_CACHE_NONE) ?
AGP_USER_MEMORY : AGP_USER_CACHED_MEMORY;
intel_gtt_insert_sg_entries(pages, start >> PAGE_SHIFT, flags);
-
}
static void i915_ggtt_clear_range(struct i915_address_space *vm,
- uint64_t start,
- uint64_t length)
+ u64 start, u64 length)
{
intel_gtt_clear_range(start >> PAGE_SHIFT, length >> PAGE_SHIFT);
}
{
struct drm_i915_private *i915 = vma->vm->i915;
struct drm_i915_gem_object *obj = vma->obj;
- u32 pte_flags = 0;
- int ret;
+ u32 pte_flags;
- ret = i915_get_ggtt_vma_pages(vma);
- if (ret)
- return ret;
+ if (unlikely(!vma->pages)) {
+ int ret = i915_get_ggtt_vma_pages(vma);
+ if (ret)
+ return ret;
+ }
/* Currently applicable only to VLV */
+ pte_flags = 0;
if (obj->gt_ro)
pte_flags |= PTE_READ_ONLY;
return 0;
}
+static void ggtt_unbind_vma(struct i915_vma *vma)
+{
+ struct drm_i915_private *i915 = vma->vm->i915;
+
+ intel_runtime_pm_get(i915);
+ vma->vm->clear_range(vma->vm, vma->node.start, vma->size);
+ intel_runtime_pm_put(i915);
+}
+
static int aliasing_gtt_bind_vma(struct i915_vma *vma,
enum i915_cache_level cache_level,
u32 flags)
u32 pte_flags;
int ret;
- ret = i915_get_ggtt_vma_pages(vma);
- if (ret)
- return ret;
+ if (unlikely(!vma->pages)) {
+ ret = i915_get_ggtt_vma_pages(vma);
+ if (ret)
+ return ret;
+ }
/* Currently applicable only to VLV */
pte_flags = 0;
if (vma->obj->gt_ro)
pte_flags |= PTE_READ_ONLY;
+ if (flags & I915_VMA_LOCAL_BIND) {
+ struct i915_hw_ppgtt *appgtt = i915->mm.aliasing_ppgtt;
+
+ if (appgtt->base.allocate_va_range) {
+ ret = appgtt->base.allocate_va_range(&appgtt->base,
+ vma->node.start,
+ vma->node.size);
+ if (ret)
+ goto err_pages;
+ }
+
+ appgtt->base.insert_entries(&appgtt->base,
+ vma->pages, vma->node.start,
+ cache_level, pte_flags);
+ }
if (flags & I915_VMA_GLOBAL_BIND) {
intel_runtime_pm_get(i915);
intel_runtime_pm_put(i915);
}
- if (flags & I915_VMA_LOCAL_BIND) {
- struct i915_hw_ppgtt *appgtt = i915->mm.aliasing_ppgtt;
- appgtt->base.insert_entries(&appgtt->base,
- vma->pages, vma->node.start,
- cache_level, pte_flags);
- }
-
return 0;
+
+err_pages:
+ if (!(vma->flags & (I915_VMA_GLOBAL_BIND | I915_VMA_LOCAL_BIND))) {
+ if (vma->pages != vma->obj->mm.pages) {
+ GEM_BUG_ON(!vma->pages);
+ sg_free_table(vma->pages);
+ kfree(vma->pages);
+ }
+ vma->pages = NULL;
+ }
+ return ret;
}
-static void ggtt_unbind_vma(struct i915_vma *vma)
+static void aliasing_gtt_unbind_vma(struct i915_vma *vma)
{
struct drm_i915_private *i915 = vma->vm->i915;
- struct i915_hw_ppgtt *appgtt = i915->mm.aliasing_ppgtt;
- const u64 size = min(vma->size, vma->node.size);
if (vma->flags & I915_VMA_GLOBAL_BIND) {
intel_runtime_pm_get(i915);
- vma->vm->clear_range(vma->vm,
- vma->node.start, size);
+ vma->vm->clear_range(vma->vm, vma->node.start, vma->size);
intel_runtime_pm_put(i915);
}
- if (vma->flags & I915_VMA_LOCAL_BIND && appgtt)
- appgtt->base.clear_range(&appgtt->base,
- vma->node.start, size);
+ if (vma->flags & I915_VMA_LOCAL_BIND) {
+ struct i915_address_space *vm = &i915->mm.aliasing_ppgtt->base;
+
+ vm->clear_range(vm, vma->node.start, vma->size);
+ }
}
void i915_gem_gtt_finish_pages(struct drm_i915_gem_object *obj,
u64 *start,
u64 *end)
{
- if (node->color != color)
+ if (node->allocated && node->color != color)
*start += I915_GTT_PAGE_SIZE;
+ /* Also leave a space between the unallocated reserved node after the
+ * GTT and any objects within the GTT, i.e. we use the color adjustment
+ * to insert a guard page to prevent prefetches crossing over the
+ * GTT boundary.
+ */
node = list_next_entry(node, node_list);
- if (node->allocated && node->color != color)
+ if (node->color != color)
*end -= I915_GTT_PAGE_SIZE;
}
+int i915_gem_init_aliasing_ppgtt(struct drm_i915_private *i915)
+{
+ struct i915_ggtt *ggtt = &i915->ggtt;
+ struct i915_hw_ppgtt *ppgtt;
+ int err;
+
+ ppgtt = i915_ppgtt_create(i915, ERR_PTR(-EPERM), "[alias]");
+ if (IS_ERR(ppgtt))
+ return PTR_ERR(ppgtt);
+
+ if (WARN_ON(ppgtt->base.total < ggtt->base.total)) {
+ err = -ENODEV;
+ goto err_ppgtt;
+ }
+
+ if (ppgtt->base.allocate_va_range) {
+ /* Note we only pre-allocate as far as the end of the global
+ * GTT. On 48b / 4-level page-tables, the difference is very,
+ * very significant! We have to preallocate as GVT/vgpu does
+ * not like the page directory disappearing.
+ */
+ err = ppgtt->base.allocate_va_range(&ppgtt->base,
+ 0, ggtt->base.total);
+ if (err)
+ goto err_ppgtt;
+ }
+
+ i915->mm.aliasing_ppgtt = ppgtt;
+
+ WARN_ON(ggtt->base.bind_vma != ggtt_bind_vma);
+ ggtt->base.bind_vma = aliasing_gtt_bind_vma;
+
+ WARN_ON(ggtt->base.unbind_vma != ggtt_unbind_vma);
+ ggtt->base.unbind_vma = aliasing_gtt_unbind_vma;
+
+ return 0;
+
+err_ppgtt:
+ i915_ppgtt_put(ppgtt);
+ return err;
+}
+
+void i915_gem_fini_aliasing_ppgtt(struct drm_i915_private *i915)
+{
+ struct i915_ggtt *ggtt = &i915->ggtt;
+ struct i915_hw_ppgtt *ppgtt;
+
+ ppgtt = fetch_and_zero(&i915->mm.aliasing_ppgtt);
+ if (!ppgtt)
+ return;
+
+ i915_ppgtt_put(ppgtt);
+
+ ggtt->base.bind_vma = ggtt_bind_vma;
+ ggtt->base.unbind_vma = ggtt_unbind_vma;
+}
+
int i915_gem_init_ggtt(struct drm_i915_private *dev_priv)
{
/* Let GEM Manage all of the aperture.
*/
struct i915_ggtt *ggtt = &dev_priv->ggtt;
unsigned long hole_start, hole_end;
- struct i915_hw_ppgtt *ppgtt;
struct drm_mm_node *entry;
int ret;
ggtt->base.total - PAGE_SIZE, PAGE_SIZE);
if (USES_PPGTT(dev_priv) && !USES_FULL_PPGTT(dev_priv)) {
- ppgtt = kzalloc(sizeof(*ppgtt), GFP_KERNEL);
- if (!ppgtt) {
- ret = -ENOMEM;
- goto err;
- }
-
- ret = __hw_ppgtt_init(ppgtt, dev_priv);
+ ret = i915_gem_init_aliasing_ppgtt(dev_priv);
if (ret)
- goto err_ppgtt;
-
- if (ppgtt->base.allocate_va_range) {
- ret = ppgtt->base.allocate_va_range(&ppgtt->base, 0,
- ppgtt->base.total);
- if (ret)
- goto err_ppgtt_cleanup;
- }
-
- ppgtt->base.clear_range(&ppgtt->base,
- ppgtt->base.start,
- ppgtt->base.total);
-
- dev_priv->mm.aliasing_ppgtt = ppgtt;
- WARN_ON(ggtt->base.bind_vma != ggtt_bind_vma);
- ggtt->base.bind_vma = aliasing_gtt_bind_vma;
+ goto err;
}
return 0;
-err_ppgtt_cleanup:
- ppgtt->base.cleanup(&ppgtt->base);
-err_ppgtt:
- kfree(ppgtt);
err:
drm_mm_remove_node(&ggtt->error_capture);
return ret;
void i915_ggtt_cleanup_hw(struct drm_i915_private *dev_priv)
{
struct i915_ggtt *ggtt = &dev_priv->ggtt;
+ struct i915_vma *vma, *vn;
- if (dev_priv->mm.aliasing_ppgtt) {
- struct i915_hw_ppgtt *ppgtt = dev_priv->mm.aliasing_ppgtt;
- ppgtt->base.cleanup(&ppgtt->base);
- kfree(ppgtt);
- }
+ ggtt->base.closed = true;
+
+ mutex_lock(&dev_priv->drm.struct_mutex);
+ WARN_ON(!list_empty(&ggtt->base.active_list));
+ list_for_each_entry_safe(vma, vn, &ggtt->base.inactive_list, vm_link)
+ WARN_ON(i915_vma_unbind(vma));
+ mutex_unlock(&dev_priv->drm.struct_mutex);
i915_gem_cleanup_stolen(&dev_priv->drm);
+ mutex_lock(&dev_priv->drm.struct_mutex);
+ i915_gem_fini_aliasing_ppgtt(dev_priv);
+
if (drm_mm_node_allocated(&ggtt->error_capture))
drm_mm_remove_node(&ggtt->error_capture);
if (drm_mm_initialized(&ggtt->base.mm)) {
intel_vgt_deballoon(dev_priv);
-
- mutex_lock(&dev_priv->drm.struct_mutex);
i915_address_space_fini(&ggtt->base);
- mutex_unlock(&dev_priv->drm.struct_mutex);
}
ggtt->base.cleanup(&ggtt->base);
+ mutex_unlock(&dev_priv->drm.struct_mutex);
arch_phys_wc_del(ggtt->mtrr);
io_mapping_fini(&ggtt->mappable);
return -ENOMEM;
}
- ret = setup_scratch_page(dev_priv, &ggtt->base.scratch_page, GFP_DMA32);
+ ret = setup_scratch_page(&ggtt->base, GFP_DMA32);
if (ret) {
DRM_ERROR("Scratch setup failed\n");
/* iounmap will also get called at remove, but meh */
* writing this data shouldn't be harmful even in those cases. */
static void bdw_setup_private_ppat(struct drm_i915_private *dev_priv)
{
- uint64_t pat;
+ u64 pat;
pat = GEN8_PPAT(0, GEN8_PPAT_WB | GEN8_PPAT_LLC) | /* for normal objects, no eLLC */
GEN8_PPAT(1, GEN8_PPAT_WC | GEN8_PPAT_LLCELLC) | /* for something pointing to ptes? */
static void chv_setup_private_ppat(struct drm_i915_private *dev_priv)
{
- uint64_t pat;
+ u64 pat;
/*
* Map WB on BDW to snooped on CHV.
struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm);
iounmap(ggtt->gsm);
- cleanup_scratch_page(vm->i915, &vm->scratch_page);
+ cleanup_scratch_page(vm);
}
static int gen8_gmch_probe(struct i915_ggtt *ggtt)
ggtt->base.clear_range = gen8_ggtt_clear_range;
ggtt->base.insert_entries = gen8_ggtt_insert_entries;
- if (IS_CHERRYVIEW(dev_priv))
- ggtt->base.insert_entries = gen8_ggtt_insert_entries__BKL;
ggtt->invalidate = gen6_ggtt_invalidate;
int ret;
ggtt->base.i915 = dev_priv;
+ ggtt->base.dma = &dev_priv->drm.pdev->dev;
if (INTEL_GEN(dev_priv) <= 5)
ret = i915_gmch_probe(ggtt);
INIT_LIST_HEAD(&dev_priv->vm_list);
- /* Subtract the guard page before address space initialization to
- * shrink the range used by drm_mm.
+ /* Note that we use page colouring to enforce a guard page at the
+ * end of the address space. This is required as the CS may prefetch
+ * beyond the end of the batch buffer, across the page boundary,
+ * and beyond the end of the GTT if we do not provide a guard.
*/
mutex_lock(&dev_priv->drm.struct_mutex);
- ggtt->base.total -= PAGE_SIZE;
i915_address_space_init(&ggtt->base, dev_priv, "[global]");
- ggtt->base.total += PAGE_SIZE;
- if (!HAS_LLC(dev_priv))
+ if (!HAS_LLC(dev_priv) && !USES_PPGTT(dev_priv))
ggtt->base.mm.color_adjust = i915_gtt_color_adjust;
mutex_unlock(&dev_priv->drm.struct_mutex);
i915_check_and_clear_faults(dev_priv);
/* First fill our portion of the GTT with scratch pages */
- ggtt->base.clear_range(&ggtt->base, ggtt->base.start, ggtt->base.total);
+ ggtt->base.clear_range(&ggtt->base, 0, ggtt->base.total);
ggtt->base.closed = true; /* skip rewriting PTE on VMA unbind */
struct i915_address_space *vm;
list_for_each_entry(vm, &dev_priv->vm_list, global_link) {
- /* TODO: Perhaps it shouldn't be gen6 specific */
-
struct i915_hw_ppgtt *ppgtt;
if (i915_is_ggtt(vm))
else
ppgtt = i915_vm_to_ppgtt(vm);
- gen6_write_page_range(dev_priv, &ppgtt->pd,
- 0, ppgtt->base.total);
+ gen6_write_page_range(ppgtt, 0, ppgtt->base.total);
}
}
return sg;
}
-static struct sg_table *
-intel_rotate_fb_obj_pages(const struct intel_rotation_info *rot_info,
- struct drm_i915_gem_object *obj)
+static noinline struct sg_table *
+intel_rotate_pages(struct intel_rotation_info *rot_info,
+ struct drm_i915_gem_object *obj)
{
- const size_t n_pages = obj->base.size / PAGE_SIZE;
+ const unsigned long n_pages = obj->base.size / PAGE_SIZE;
unsigned int size = intel_rotation_info_size(rot_info);
struct sgt_iter sgt_iter;
dma_addr_t dma_addr;
return ERR_PTR(ret);
}
-static struct sg_table *
+static noinline struct sg_table *
intel_partial_pages(const struct i915_ggtt_view *view,
struct drm_i915_gem_object *obj)
{
static int
i915_get_ggtt_vma_pages(struct i915_vma *vma)
{
- int ret = 0;
+ int ret;
/* The vma->pages are only valid within the lifespan of the borrowed
* obj->mm.pages. When the obj->mm.pages sg_table is regenerated, so
*/
GEM_BUG_ON(!i915_gem_object_has_pinned_pages(vma->obj));
- if (vma->pages)
+ switch (vma->ggtt_view.type) {
+ case I915_GGTT_VIEW_NORMAL:
+ vma->pages = vma->obj->mm.pages;
return 0;
- if (vma->ggtt_view.type == I915_GGTT_VIEW_NORMAL)
- vma->pages = vma->obj->mm.pages;
- else if (vma->ggtt_view.type == I915_GGTT_VIEW_ROTATED)
+ case I915_GGTT_VIEW_ROTATED:
vma->pages =
- intel_rotate_fb_obj_pages(&vma->ggtt_view.rotated,
- vma->obj);
- else if (vma->ggtt_view.type == I915_GGTT_VIEW_PARTIAL)
+ intel_rotate_pages(&vma->ggtt_view.rotated, vma->obj);
+ break;
+
+ case I915_GGTT_VIEW_PARTIAL:
vma->pages = intel_partial_pages(&vma->ggtt_view, vma->obj);
- else
+ break;
+
+ default:
WARN_ONCE(1, "GGTT view %u not implemented!\n",
vma->ggtt_view.type);
+ return -EINVAL;
+ }
- if (!vma->pages) {
- DRM_ERROR("Failed to get pages for GGTT view type %u!\n",
- vma->ggtt_view.type);
- ret = -EINVAL;
- } else if (IS_ERR(vma->pages)) {
+ ret = 0;
+ if (unlikely(IS_ERR(vma->pages))) {
ret = PTR_ERR(vma->pages);
vma->pages = NULL;
DRM_ERROR("Failed to get pages for VMA view type %u (%d)!\n",
vma->ggtt_view.type, ret);
}
-
return ret;
}
size, alignment, color,
start, end, DRM_MM_INSERT_EVICT);
}
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/mock_gtt.c"
+#include "selftests/i915_gem_gtt.c"
+#endif
#include <linux/io-mapping.h>
#include <linux/mm.h>
+#include <linux/pagevec.h>
#include "i915_gem_timeline.h"
#include "i915_gem_request.h"
+#include "i915_selftest.h"
#define I915_GTT_PAGE_SIZE 4096UL
#define I915_GTT_MIN_ALIGNMENT I915_GTT_PAGE_SIZE
struct drm_i915_file_private;
struct drm_i915_fence_reg;
-typedef uint32_t gen6_pte_t;
-typedef uint64_t gen8_pte_t;
-typedef uint64_t gen8_pde_t;
-typedef uint64_t gen8_ppgtt_pdpe_t;
-typedef uint64_t gen8_ppgtt_pml4e_t;
+typedef u32 gen6_pte_t;
+typedef u64 gen8_pte_t;
+typedef u64 gen8_pde_t;
+typedef u64 gen8_ppgtt_pdpe_t;
+typedef u64 gen8_ppgtt_pml4e_t;
#define ggtt_total_entries(ggtt) ((ggtt)->base.total >> PAGE_SHIFT)
#define GEN6_PTE_UNCACHED (1 << 1)
#define GEN6_PTE_VALID (1 << 0)
-#define I915_PTES(pte_len) (PAGE_SIZE / (pte_len))
+#define I915_PTES(pte_len) ((unsigned int)(PAGE_SIZE / (pte_len)))
#define I915_PTE_MASK(pte_len) (I915_PTES(pte_len) - 1)
#define I915_PDES 512
#define I915_PDE_MASK (I915_PDES - 1)
#define HSW_GTT_ADDR_ENCODE(addr) ((addr) | (((addr) >> 28) & 0x7f0))
#define HSW_PTE_ADDR_ENCODE(addr) HSW_GTT_ADDR_ENCODE(addr)
-/* GEN8 legacy style address is defined as a 3 level page table:
+/* GEN8 32b style address is defined as a 3 level page table:
* 31:30 | 29:21 | 20:12 | 11:0
* PDPE | PDE | PTE | offset
* The difference as compared to normal x86 3 level page table is the PDPEs are
* programmed via register.
- *
- * GEN8 48b legacy style address is defined as a 4 level page table:
+ */
+#define GEN8_3LVL_PDPES 4
+#define GEN8_PDE_SHIFT 21
+#define GEN8_PDE_MASK 0x1ff
+#define GEN8_PTE_SHIFT 12
+#define GEN8_PTE_MASK 0x1ff
+#define GEN8_PTES I915_PTES(sizeof(gen8_pte_t))
+
+/* GEN8 48b style address is defined as a 4 level page table:
* 47:39 | 38:30 | 29:21 | 20:12 | 11:0
* PML4E | PDPE | PDE | PTE | offset
*/
/* NB: GEN8_PDPE_MASK is untrue for 32b platforms, but it has no impact on 32b page
* tables */
#define GEN8_PDPE_MASK 0x1ff
-#define GEN8_PDE_SHIFT 21
-#define GEN8_PDE_MASK 0x1ff
-#define GEN8_PTE_SHIFT 12
-#define GEN8_PTE_MASK 0x1ff
-#define GEN8_LEGACY_PDPES 4
-#define GEN8_PTES I915_PTES(sizeof(gen8_pte_t))
-
-#define I915_PDPES_PER_PDP(dev_priv) (USES_FULL_48BIT_PPGTT(dev_priv) ?\
- GEN8_PML4ES_PER_PML4 : GEN8_LEGACY_PDPES)
#define PPAT_UNCACHED_INDEX (_PAGE_PWT | _PAGE_PCD)
#define PPAT_CACHED_PDE_INDEX 0 /* WB LLC */
#define GEN8_PPAT_WC (1<<0)
#define GEN8_PPAT_UC (0<<0)
#define GEN8_PPAT_ELLC_OVERRIDE (0<<2)
-#define GEN8_PPAT(i, x) ((uint64_t) (x) << ((i) * 8))
+#define GEN8_PPAT(i, x) ((u64)(x) << ((i) * 8))
struct sg_table;
/* For gen6/gen7 only. This is the offset in the GGTT
* where the page directory entries for PPGTT begin
*/
- uint32_t ggtt_offset;
+ u32 ggtt_offset;
};
};
struct i915_page_table {
struct i915_page_dma base;
-
- unsigned long *used_ptes;
+ unsigned int used_ptes;
};
struct i915_page_directory {
struct i915_page_dma base;
- unsigned long *used_pdes;
struct i915_page_table *page_table[I915_PDES]; /* PDEs */
+ unsigned int used_pdes;
};
struct i915_page_directory_pointer {
struct i915_page_dma base;
-
- unsigned long *used_pdpes;
struct i915_page_directory **page_directory;
+ unsigned int used_pdpes;
};
struct i915_pml4 {
struct i915_page_dma base;
-
- DECLARE_BITMAP(used_pml4es, GEN8_PML4ES_PER_PML4);
struct i915_page_directory_pointer *pdps[GEN8_PML4ES_PER_PML4];
};
struct drm_mm mm;
struct i915_gem_timeline timeline;
struct drm_i915_private *i915;
+ struct device *dma;
/* Every address space belongs to a struct file - except for the global
* GTT that is owned by the driver (and so @file is set to NULL). In
* principle, no information should leak from one context to another
*/
struct drm_i915_file_private *file;
struct list_head global_link;
- u64 start; /* Start offset always 0 for dri2 */
u64 total; /* size addr space maps (ex. 2GB for ggtt) */
bool closed;
*/
struct list_head unbound_list;
+ struct pagevec free_pages;
+ bool pt_kmap_wc;
+
/* FIXME: Need a more generic return type */
gen6_pte_t (*pte_encode)(dma_addr_t addr,
enum i915_cache_level level,
/* flags for pte_encode */
#define PTE_READ_ONLY (1<<0)
int (*allocate_va_range)(struct i915_address_space *vm,
- uint64_t start,
- uint64_t length);
+ u64 start, u64 length);
void (*clear_range)(struct i915_address_space *vm,
- uint64_t start,
- uint64_t length);
+ u64 start, u64 length);
void (*insert_page)(struct i915_address_space *vm,
dma_addr_t addr,
- uint64_t offset,
+ u64 offset,
enum i915_cache_level cache_level,
u32 flags);
void (*insert_entries)(struct i915_address_space *vm,
struct sg_table *st,
- uint64_t start,
- enum i915_cache_level cache_level, u32 flags);
+ u64 start,
+ enum i915_cache_level cache_level,
+ u32 flags);
void (*cleanup)(struct i915_address_space *vm);
/** Unmap an object from an address space. This usually consists of
* setting the valid PTE entries to a reserved scratch page. */
int (*bind_vma)(struct i915_vma *vma,
enum i915_cache_level cache_level,
u32 flags);
+
+ I915_SELFTEST_DECLARE(struct fault_attr fault_attr);
};
#define i915_is_ggtt(V) (!(V)->file)
+static inline bool
+i915_vm_is_48bit(const struct i915_address_space *vm)
+{
+ return (vm->total - 1) >> 32;
+}
+
/* The Graphics Translation Table is the way in which GEN hardware translates a
* Graphics Virtual Address into a Physical Address. In addition to the normal
* collateral associated with any va->pa translations GEN hardware also has a
gen6_pte_t __iomem *pd_addr;
- int (*enable)(struct i915_hw_ppgtt *ppgtt);
int (*switch_mm)(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req);
void (*debug_dump)(struct i915_hw_ppgtt *ppgtt, struct seq_file *m);
(pt = (pd)->page_table[iter], true); \
++iter)
-static inline uint32_t i915_pte_index(uint64_t address, uint32_t pde_shift)
+static inline u32 i915_pte_index(u64 address, unsigned int pde_shift)
{
- const uint32_t mask = NUM_PTE(pde_shift) - 1;
+ const u32 mask = NUM_PTE(pde_shift) - 1;
return (address >> PAGE_SHIFT) & mask;
}
* does not cross a page table boundary, so the max value would be
* GEN6_PTES for GEN6, and GEN8_PTES for GEN8.
*/
-static inline uint32_t i915_pte_count(uint64_t addr, size_t length,
- uint32_t pde_shift)
+static inline u32 i915_pte_count(u64 addr, u64 length, unsigned int pde_shift)
{
- const uint64_t mask = ~((1ULL << pde_shift) - 1);
- uint64_t end;
+ const u64 mask = ~((1ULL << pde_shift) - 1);
+ u64 end;
WARN_ON(length == 0);
WARN_ON(offset_in_page(addr|length));
return i915_pte_index(end, pde_shift) - i915_pte_index(addr, pde_shift);
}
-static inline uint32_t i915_pde_index(uint64_t addr, uint32_t shift)
+static inline u32 i915_pde_index(u64 addr, u32 shift)
{
return (addr >> shift) & I915_PDE_MASK;
}
-static inline uint32_t gen6_pte_index(uint32_t addr)
+static inline u32 gen6_pte_index(u32 addr)
{
return i915_pte_index(addr, GEN6_PDE_SHIFT);
}
-static inline size_t gen6_pte_count(uint32_t addr, uint32_t length)
+static inline u32 gen6_pte_count(u32 addr, u32 length)
{
return i915_pte_count(addr, length, GEN6_PDE_SHIFT);
}
-static inline uint32_t gen6_pde_index(uint32_t addr)
+static inline u32 gen6_pde_index(u32 addr)
{
return i915_pde_index(addr, GEN6_PDE_SHIFT);
}
+static inline unsigned int
+i915_pdpes_per_pdp(const struct i915_address_space *vm)
+{
+ if (i915_vm_is_48bit(vm))
+ return GEN8_PML4ES_PER_PML4;
+
+ return GEN8_3LVL_PDPES;
+}
+
/* Equivalent to the gen6 version, For each pde iterates over every pde
* between from start until start + length. On gen8+ it simply iterates
* over every page directory entry in a page directory.
#define gen8_for_each_pdpe(pd, pdp, start, length, iter) \
for (iter = gen8_pdpe_index(start); \
- length > 0 && iter < I915_PDPES_PER_PDP(dev) && \
+ length > 0 && iter < i915_pdpes_per_pdp(vm) && \
(pd = (pdp)->page_directory[iter], true); \
({ u64 temp = ALIGN(start+1, 1 << GEN8_PDPE_SHIFT); \
temp = min(temp - start, length); \
temp = min(temp - start, length); \
start += temp, length -= temp; }), ++iter)
-static inline uint32_t gen8_pte_index(uint64_t address)
+static inline u32 gen8_pte_index(u64 address)
{
return i915_pte_index(address, GEN8_PDE_SHIFT);
}
-static inline uint32_t gen8_pde_index(uint64_t address)
+static inline u32 gen8_pde_index(u64 address)
{
return i915_pde_index(address, GEN8_PDE_SHIFT);
}
-static inline uint32_t gen8_pdpe_index(uint64_t address)
+static inline u32 gen8_pdpe_index(u64 address)
{
return (address >> GEN8_PDPE_SHIFT) & GEN8_PDPE_MASK;
}
-static inline uint32_t gen8_pml4e_index(uint64_t address)
+static inline u32 gen8_pml4e_index(u64 address)
{
return (address >> GEN8_PML4E_SHIFT) & GEN8_PML4E_MASK;
}
-static inline size_t gen8_pte_count(uint64_t address, uint64_t length)
+static inline u64 gen8_pte_count(u64 address, u64 length)
{
return i915_pte_count(address, length, GEN8_PDE_SHIFT);
}
static inline dma_addr_t
i915_page_dir_dma_addr(const struct i915_hw_ppgtt *ppgtt, const unsigned n)
{
- return test_bit(n, ppgtt->pdp.used_pdpes) ?
- px_dma(ppgtt->pdp.page_directory[n]) :
- px_dma(ppgtt->base.scratch_pd);
+ return px_dma(ppgtt->pdp.page_directory[n]);
}
static inline struct i915_ggtt *
return container_of(vm, struct i915_ggtt, base);
}
+int i915_gem_init_aliasing_ppgtt(struct drm_i915_private *i915);
+void i915_gem_fini_aliasing_ppgtt(struct drm_i915_private *i915);
+
int i915_ggtt_probe_hw(struct drm_i915_private *dev_priv);
int i915_ggtt_init_hw(struct drm_i915_private *dev_priv);
int i915_ggtt_enable_hw(struct drm_i915_private *dev_priv);
{
struct scatterlist *sg;
- for (sg = st->sgl; sg; sg = __sg_next(sg))
- __free_pages(sg_page(sg), get_order(sg->length));
+ for (sg = st->sgl; sg; sg = __sg_next(sg)) {
+ if (sg_page(sg))
+ __free_pages(sg_page(sg), get_order(sg->length));
+ }
sg_free_table(st);
kfree(st);
return st;
err:
+ sg_set_page(sg, NULL, 0, 0);
sg_mark_end(sg);
internal_free_pages(st);
return ERR_PTR(-ENOMEM);
#include <drm/i915_drm.h>
+#include "i915_selftest.h"
+
struct drm_i915_gem_object_ops {
unsigned int flags;
#define I915_GEM_OBJECT_HAS_STRUCT_PAGE 0x1
struct list_head obj_exec_link;
struct list_head batch_pool_link;
+ I915_SELFTEST_DECLARE(struct list_head st_link);
unsigned long flags;
struct reservation_object *resv;
/** References from framebuffers, locks out tiling changes. */
- unsigned long framebuffer_references;
+ unsigned int framebuffer_references;
/** Record of address bit 17 of each page at last unbind. */
unsigned long *bit_17;
- struct i915_gem_userptr {
- uintptr_t ptr;
- unsigned read_only :1;
+ union {
+ struct i915_gem_userptr {
+ uintptr_t ptr;
+ unsigned read_only :1;
+
+ struct i915_mm_struct *mm;
+ struct i915_mmu_object *mmu_object;
+ struct work_struct *work;
+ } userptr;
- struct i915_mm_struct *mm;
- struct i915_mmu_object *mmu_object;
- struct work_struct *work;
- } userptr;
+ unsigned long scratch;
+ };
/** for phys allocated objects */
struct drm_dma_handle *phys_handle;
__deprecated
extern void drm_gem_object_unreference_unlocked(struct drm_gem_object *);
-static inline bool
-i915_gem_object_is_dead(const struct drm_i915_gem_object *obj)
+static inline void i915_gem_object_lock(struct drm_i915_gem_object *obj)
{
- return kref_read(&obj->base.refcount) == 0;
+ reservation_object_lock(obj->resv, NULL);
+}
+
+static inline void i915_gem_object_unlock(struct drm_i915_gem_object *obj)
+{
+ reservation_object_unlock(obj->resv);
}
static inline bool
void __i915_gem_object_release_unless_active(struct drm_i915_gem_object *obj);
+static inline bool
+i915_gem_object_is_framebuffer(const struct drm_i915_gem_object *obj)
+{
+ return READ_ONCE(obj->framebuffer_references);
+}
+
static inline unsigned int
i915_gem_object_get_tiling(struct drm_i915_gem_object *obj)
{
return engine;
}
+void i915_gem_object_flush_if_display(struct drm_i915_gem_object *obj);
+
#endif
* caught trying to reuse dead objects.
*/
i915_sw_fence_fini(&req->submit);
- i915_sw_fence_fini(&req->execute);
kmem_cache_free(req->i915->requests, req);
}
.release = i915_fence_release,
};
-int i915_gem_request_add_to_client(struct drm_i915_gem_request *req,
- struct drm_file *file)
-{
- struct drm_i915_private *dev_private;
- struct drm_i915_file_private *file_priv;
-
- WARN_ON(!req || !file || req->file_priv);
-
- if (!req || !file)
- return -EINVAL;
-
- if (req->file_priv)
- return -EINVAL;
-
- dev_private = req->i915;
- file_priv = file->driver_priv;
-
- spin_lock(&file_priv->mm.lock);
- req->file_priv = file_priv;
- list_add_tail(&req->client_list, &file_priv->mm.request_list);
- spin_unlock(&file_priv->mm.lock);
-
- return 0;
-}
-
static inline void
i915_gem_request_remove_from_client(struct drm_i915_gem_request *request)
{
- struct drm_i915_file_private *file_priv = request->file_priv;
+ struct drm_i915_file_private *file_priv;
+ file_priv = request->file_priv;
if (!file_priv)
return;
spin_lock(&file_priv->mm.lock);
- list_del(&request->client_list);
- request->file_priv = NULL;
+ if (request->file_priv) {
+ list_del(&request->client_link);
+ request->file_priv = NULL;
+ }
spin_unlock(&file_priv->mm.lock);
}
pt->priority = INT_MIN;
}
+static int reset_all_global_seqno(struct drm_i915_private *i915, u32 seqno)
+{
+ struct i915_gem_timeline *timeline = &i915->gt.global_timeline;
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ int ret;
+
+ /* Carefully retire all requests without writing to the rings */
+ ret = i915_gem_wait_for_idle(i915,
+ I915_WAIT_INTERRUPTIBLE |
+ I915_WAIT_LOCKED);
+ if (ret)
+ return ret;
+
+ i915_gem_retire_requests(i915);
+ GEM_BUG_ON(i915->gt.active_requests > 1);
+
+ /* If the seqno wraps around, we need to clear the breadcrumb rbtree */
+ for_each_engine(engine, i915, id) {
+ struct intel_timeline *tl = &timeline->engine[id];
+
+ if (wait_for(intel_engine_is_idle(engine), 50))
+ return -EBUSY;
+
+ if (!i915_seqno_passed(seqno, tl->seqno)) {
+ /* spin until threads are complete */
+ while (intel_breadcrumbs_busy(engine))
+ cond_resched();
+ }
+
+ /* Finally reset hw state */
+ tl->seqno = seqno;
+ intel_engine_init_global_seqno(engine, seqno);
+ }
+
+ list_for_each_entry(timeline, &i915->gt.timelines, link) {
+ for_each_engine(engine, i915, id) {
+ struct intel_timeline *tl = &timeline->engine[id];
+
+ memset(tl->sync_seqno, 0, sizeof(tl->sync_seqno));
+ }
+ }
+
+ return 0;
+}
+
+int i915_gem_set_global_seqno(struct drm_device *dev, u32 seqno)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+
+ lockdep_assert_held(&dev_priv->drm.struct_mutex);
+
+ if (seqno == 0)
+ return -EINVAL;
+
+ /* HWS page needs to be set less than what we
+ * will inject to ring
+ */
+ return reset_all_global_seqno(dev_priv, seqno - 1);
+}
+
+static int reserve_seqno(struct intel_engine_cs *engine)
+{
+ u32 active = ++engine->timeline->inflight_seqnos;
+ u32 seqno = engine->timeline->seqno;
+ int ret;
+
+ /* Reservation is fine until we need to wrap around */
+ if (likely(!add_overflows(seqno, active)))
+ return 0;
+
+ ret = reset_all_global_seqno(engine->i915, 0);
+ if (ret) {
+ engine->timeline->inflight_seqnos--;
+ return ret;
+ }
+
+ return 0;
+}
+
+static void unreserve_seqno(struct intel_engine_cs *engine)
+{
+ GEM_BUG_ON(!engine->timeline->inflight_seqnos);
+ engine->timeline->inflight_seqnos--;
+}
+
void i915_gem_retire_noop(struct i915_gem_active *active,
struct drm_i915_gem_request *request)
{
lockdep_assert_held(&request->i915->drm.struct_mutex);
GEM_BUG_ON(!i915_sw_fence_signaled(&request->submit));
- GEM_BUG_ON(!i915_sw_fence_signaled(&request->execute));
GEM_BUG_ON(!i915_gem_request_completed(request));
GEM_BUG_ON(!request->i915->gt.active_requests);
&request->i915->gt.idle_work,
msecs_to_jiffies(100));
}
+ unreserve_seqno(request->engine);
/* Walk through the active list, calling retire on each. This allows
* objects to track their GPU activity and mark themselves as idle
} while (tmp != req);
}
-static int i915_gem_init_global_seqno(struct drm_i915_private *i915, u32 seqno)
-{
- struct i915_gem_timeline *timeline = &i915->gt.global_timeline;
- struct intel_engine_cs *engine;
- enum intel_engine_id id;
- int ret;
-
- /* Carefully retire all requests without writing to the rings */
- ret = i915_gem_wait_for_idle(i915,
- I915_WAIT_INTERRUPTIBLE |
- I915_WAIT_LOCKED);
- if (ret)
- return ret;
-
- i915_gem_retire_requests(i915);
- GEM_BUG_ON(i915->gt.active_requests > 1);
-
- /* If the seqno wraps around, we need to clear the breadcrumb rbtree */
- if (!i915_seqno_passed(seqno, atomic_read(&timeline->seqno))) {
- while (intel_breadcrumbs_busy(i915))
- cond_resched(); /* spin until threads are complete */
- }
- atomic_set(&timeline->seqno, seqno);
-
- /* Finally reset hw state */
- for_each_engine(engine, i915, id)
- intel_engine_init_global_seqno(engine, seqno);
-
- list_for_each_entry(timeline, &i915->gt.timelines, link) {
- for_each_engine(engine, i915, id) {
- struct intel_timeline *tl = &timeline->engine[id];
-
- memset(tl->sync_seqno, 0, sizeof(tl->sync_seqno));
- }
- }
-
- return 0;
-}
-
-int i915_gem_set_global_seqno(struct drm_device *dev, u32 seqno)
+static u32 timeline_get_seqno(struct intel_timeline *tl)
{
- struct drm_i915_private *dev_priv = to_i915(dev);
-
- lockdep_assert_held(&dev_priv->drm.struct_mutex);
-
- if (seqno == 0)
- return -EINVAL;
-
- /* HWS page needs to be set less than what we
- * will inject to ring
- */
- return i915_gem_init_global_seqno(dev_priv, seqno - 1);
-}
-
-static int reserve_global_seqno(struct drm_i915_private *i915)
-{
- u32 active_requests = ++i915->gt.active_requests;
- u32 seqno = atomic_read(&i915->gt.global_timeline.seqno);
- int ret;
-
- /* Reservation is fine until we need to wrap around */
- if (likely(seqno + active_requests > seqno))
- return 0;
-
- ret = i915_gem_init_global_seqno(i915, 0);
- if (ret) {
- i915->gt.active_requests--;
- return ret;
- }
-
- return 0;
-}
-
-static u32 __timeline_get_seqno(struct i915_gem_timeline *tl)
-{
- /* seqno only incremented under a mutex */
- return ++tl->seqno.counter;
-}
-
-static u32 timeline_get_seqno(struct i915_gem_timeline *tl)
-{
- return atomic_inc_return(&tl->seqno);
+ return ++tl->seqno;
}
void __i915_gem_request_submit(struct drm_i915_gem_request *request)
struct intel_timeline *timeline;
u32 seqno;
+ GEM_BUG_ON(!irqs_disabled());
+ lockdep_assert_held(&engine->timeline->lock);
+
+ trace_i915_gem_request_execute(request);
+
/* Transfer from per-context onto the global per-engine timeline */
timeline = engine->timeline;
GEM_BUG_ON(timeline == request->timeline);
- assert_spin_locked(&timeline->lock);
- seqno = timeline_get_seqno(timeline->common);
+ seqno = timeline_get_seqno(timeline);
GEM_BUG_ON(!seqno);
GEM_BUG_ON(i915_seqno_passed(intel_engine_get_seqno(engine), seqno));
- GEM_BUG_ON(i915_seqno_passed(timeline->last_submitted_seqno, seqno));
- request->previous_seqno = timeline->last_submitted_seqno;
- timeline->last_submitted_seqno = seqno;
-
/* We may be recursing from the signal callback of another i915 fence */
spin_lock_nested(&request->lock, SINGLE_DEPTH_NESTING);
request->global_seqno = seqno;
intel_engine_enable_signaling(request);
spin_unlock(&request->lock);
- GEM_BUG_ON(!request->global_seqno);
engine->emit_breadcrumb(request,
request->ring->vaddr + request->postfix);
list_move_tail(&request->link, &timeline->requests);
spin_unlock(&request->timeline->lock);
- i915_sw_fence_commit(&request->execute);
+ wake_up_all(&request->execute);
}
void i915_gem_request_submit(struct drm_i915_gem_request *request)
spin_unlock_irqrestore(&engine->timeline->lock, flags);
}
-static int __i915_sw_fence_call
-submit_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
+void __i915_gem_request_unsubmit(struct drm_i915_gem_request *request)
{
- struct drm_i915_gem_request *request =
- container_of(fence, typeof(*request), submit);
+ struct intel_engine_cs *engine = request->engine;
+ struct intel_timeline *timeline;
- switch (state) {
- case FENCE_COMPLETE:
- request->engine->submit_request(request);
- break;
+ GEM_BUG_ON(!irqs_disabled());
+ lockdep_assert_held(&engine->timeline->lock);
- case FENCE_FREE:
- i915_gem_request_put(request);
- break;
- }
+ /* Only unwind in reverse order, required so that the per-context list
+ * is kept in seqno/ring order.
+ */
+ GEM_BUG_ON(request->global_seqno != engine->timeline->seqno);
+ engine->timeline->seqno--;
- return NOTIFY_DONE;
+ /* We may be recursing from the signal callback of another i915 fence */
+ spin_lock_nested(&request->lock, SINGLE_DEPTH_NESTING);
+ request->global_seqno = 0;
+ if (test_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT, &request->fence.flags))
+ intel_engine_cancel_signaling(request);
+ spin_unlock(&request->lock);
+
+ /* Transfer back from the global per-engine timeline to per-context */
+ timeline = request->timeline;
+ GEM_BUG_ON(timeline == engine->timeline);
+
+ spin_lock(&timeline->lock);
+ list_move(&request->link, &timeline->requests);
+ spin_unlock(&timeline->lock);
+
+ /* We don't need to wake_up any waiters on request->execute, they
+ * will get woken by any other event or us re-adding this request
+ * to the engine timeline (__i915_gem_request_submit()). The waiters
+ * should be quite adapt at finding that the request now has a new
+ * global_seqno to the one they went to sleep on.
+ */
+}
+
+void i915_gem_request_unsubmit(struct drm_i915_gem_request *request)
+{
+ struct intel_engine_cs *engine = request->engine;
+ unsigned long flags;
+
+ /* Will be called from irq-context when using foreign fences. */
+ spin_lock_irqsave(&engine->timeline->lock, flags);
+
+ __i915_gem_request_unsubmit(request);
+
+ spin_unlock_irqrestore(&engine->timeline->lock, flags);
}
static int __i915_sw_fence_call
-execute_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
+submit_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
{
struct drm_i915_gem_request *request =
- container_of(fence, typeof(*request), execute);
+ container_of(fence, typeof(*request), submit);
switch (state) {
case FENCE_COMPLETE:
+ trace_i915_gem_request_submit(request);
+ request->engine->submit_request(request);
break;
case FENCE_FREE:
if (ret)
return ERR_PTR(ret);
- ret = reserve_global_seqno(dev_priv);
+ ret = reserve_seqno(engine);
if (ret)
goto err_unpin;
/* Move the oldest request to the slab-cache (if not in use!) */
req = list_first_entry_or_null(&engine->timeline->requests,
typeof(*req), link);
- if (req && __i915_gem_request_completed(req))
+ if (req && i915_gem_request_completed(req))
i915_gem_request_retire(req);
/* Beware: Dragons be flying overhead.
&i915_fence_ops,
&req->lock,
req->timeline->fence_context,
- __timeline_get_seqno(req->timeline->common));
+ timeline_get_seqno(req->timeline));
/* We bump the ref for the fence chain */
i915_sw_fence_init(&i915_gem_request_get(req)->submit, submit_notify);
- i915_sw_fence_init(&i915_gem_request_get(req)->execute, execute_notify);
-
- /* Ensure that the execute fence completes after the submit fence -
- * as we complete the execute fence from within the submit fence
- * callback, its completion would otherwise be visible first.
- */
- i915_sw_fence_await_sw_fence(&req->execute, &req->submit, &req->execq);
+ init_waitqueue_head(&req->execute);
i915_priotree_init(&req->priotree);
*/
req->head = req->ring->tail;
+ /* Check that we didn't interrupt ourselves with a new request */
+ GEM_BUG_ON(req->timeline->seqno != req->fence.seqno);
return req;
err_ctx:
kmem_cache_free(dev_priv->requests, req);
err_unreserve:
- dev_priv->gt.active_requests--;
+ unreserve_seqno(engine);
err_unpin:
engine->context_unpin(engine, ctx);
return ERR_PTR(ret);
i915_gem_request_await_request(struct drm_i915_gem_request *to,
struct drm_i915_gem_request *from)
{
+ u32 seqno;
int ret;
GEM_BUG_ON(to == from);
return ret < 0 ? ret : 0;
}
- if (!from->global_seqno) {
+ seqno = i915_gem_request_global_seqno(from);
+ if (!seqno) {
ret = i915_sw_fence_await_dma_fence(&to->submit,
&from->fence, 0,
GFP_KERNEL);
return ret < 0 ? ret : 0;
}
- if (from->global_seqno <= to->timeline->sync_seqno[from->engine->id])
+ if (seqno <= to->timeline->sync_seqno[from->engine->id])
return 0;
trace_i915_gem_ring_sync_to(to, from);
return ret;
}
- to->timeline->sync_seqno[from->engine->id] = from->global_seqno;
+ to->timeline->sync_seqno[from->engine->id] = seqno;
return 0;
}
struct intel_ring *ring = request->ring;
struct intel_timeline *timeline = request->timeline;
struct drm_i915_gem_request *prev;
+ u32 *cs;
int err;
lockdep_assert_held(&request->i915->drm.struct_mutex);
* our i915_gem_request_alloc() and called __i915_add_request() before
* us, the timeline will hold its seqno which is later than ours.
*/
- GEM_BUG_ON(i915_seqno_passed(timeline->last_submitted_seqno,
- request->fence.seqno));
+ GEM_BUG_ON(timeline->seqno != request->fence.seqno);
/*
* To ensure that this call will not fail, space for its emissions
* GPU processing the request, we never over-estimate the
* position of the ring's HEAD.
*/
- err = intel_ring_begin(request, engine->emit_breadcrumb_sz);
- GEM_BUG_ON(err);
- request->postfix = ring->tail;
- ring->tail += engine->emit_breadcrumb_sz * sizeof(u32);
+ cs = intel_ring_begin(request, engine->emit_breadcrumb_sz);
+ GEM_BUG_ON(IS_ERR(cs));
+ request->postfix = intel_ring_offset(request, cs);
/* Seal the request and mark it as pending execution. Note that
* we may inspect this state, without holding any locks, during
list_add_tail(&request->link, &timeline->requests);
spin_unlock_irq(&timeline->lock);
- GEM_BUG_ON(i915_seqno_passed(timeline->last_submitted_seqno,
- request->fence.seqno));
-
- timeline->last_submitted_seqno = request->fence.seqno;
+ GEM_BUG_ON(timeline->seqno != request->fence.seqno);
i915_gem_active_set(&timeline->last_request, request);
list_add_tail(&request->ring_link, &ring->request_list);
request->emitted_jiffies = jiffies;
- i915_gem_mark_busy(engine);
+ if (!request->i915->gt.active_requests++)
+ i915_gem_mark_busy(engine);
/* Let the backend know a new request has arrived that may need
* to adjust the existing execution schedule due to a high priority
local_bh_enable(); /* Kick the execlists tasklet if just scheduled */
}
-static void reset_wait_queue(wait_queue_head_t *q, wait_queue_t *wait)
-{
- unsigned long flags;
-
- spin_lock_irqsave(&q->lock, flags);
- if (list_empty(&wait->task_list))
- __add_wait_queue(q, wait);
- spin_unlock_irqrestore(&q->lock, flags);
-}
-
static unsigned long local_clock_us(unsigned int *cpu)
{
unsigned long t;
}
bool __i915_spin_request(const struct drm_i915_gem_request *req,
- int state, unsigned long timeout_us)
+ u32 seqno, int state, unsigned long timeout_us)
{
- unsigned int cpu;
+ struct intel_engine_cs *engine = req->engine;
+ unsigned int irq, cpu;
/* When waiting for high frequency requests, e.g. during synchronous
* rendering split between the CPU and GPU, the finite amount of time
* takes to sleep on a request, on the order of a microsecond.
*/
+ irq = atomic_read(&engine->irq_count);
timeout_us += local_clock_us(&cpu);
do {
- if (__i915_gem_request_completed(req))
+ if (seqno != i915_gem_request_global_seqno(req))
+ break;
+
+ if (i915_seqno_passed(intel_engine_get_seqno(req->engine),
+ seqno))
return true;
+ /* Seqno are meant to be ordered *before* the interrupt. If
+ * we see an interrupt without a corresponding seqno advance,
+ * assume we won't see one in the near future but require
+ * the engine->seqno_barrier() to fixup coherency.
+ */
+ if (atomic_read(&engine->irq_count) != irq)
+ break;
+
if (signal_pending_state(state, current))
break;
return false;
}
-static long
-__i915_request_wait_for_execute(struct drm_i915_gem_request *request,
- unsigned int flags,
- long timeout)
+static bool __i915_wait_request_check_and_reset(struct drm_i915_gem_request *request)
{
- const int state = flags & I915_WAIT_INTERRUPTIBLE ?
- TASK_INTERRUPTIBLE : TASK_UNINTERRUPTIBLE;
- wait_queue_head_t *q = &request->i915->gpu_error.wait_queue;
- DEFINE_WAIT(reset);
- DEFINE_WAIT(wait);
-
- if (flags & I915_WAIT_LOCKED)
- add_wait_queue(q, &reset);
-
- do {
- prepare_to_wait(&request->execute.wait, &wait, state);
-
- if (i915_sw_fence_done(&request->execute))
- break;
-
- if (flags & I915_WAIT_LOCKED &&
- i915_reset_in_progress(&request->i915->gpu_error)) {
- __set_current_state(TASK_RUNNING);
- i915_reset(request->i915);
- reset_wait_queue(q, &reset);
- continue;
- }
-
- if (signal_pending_state(state, current)) {
- timeout = -ERESTARTSYS;
- break;
- }
-
- if (!timeout) {
- timeout = -ETIME;
- break;
- }
-
- timeout = io_schedule_timeout(timeout);
- } while (1);
- finish_wait(&request->execute.wait, &wait);
-
- if (flags & I915_WAIT_LOCKED)
- remove_wait_queue(q, &reset);
+ if (likely(!i915_reset_handoff(&request->i915->gpu_error)))
+ return false;
- return timeout;
+ __set_current_state(TASK_RUNNING);
+ i915_reset(request->i915);
+ return true;
}
/**
{
const int state = flags & I915_WAIT_INTERRUPTIBLE ?
TASK_INTERRUPTIBLE : TASK_UNINTERRUPTIBLE;
- DEFINE_WAIT(reset);
+ wait_queue_head_t *errq = &req->i915->gpu_error.wait_queue;
+ DEFINE_WAIT_FUNC(reset, default_wake_function);
+ DEFINE_WAIT_FUNC(exec, default_wake_function);
struct intel_wait wait;
might_sleep();
if (!timeout)
return -ETIME;
- trace_i915_gem_request_wait_begin(req);
+ trace_i915_gem_request_wait_begin(req, flags);
+
+ add_wait_queue(&req->execute, &exec);
+ if (flags & I915_WAIT_LOCKED)
+ add_wait_queue(errq, &reset);
+
+ intel_wait_init(&wait, req);
+
+restart:
+ do {
+ set_current_state(state);
+ if (intel_wait_update_request(&wait, req))
+ break;
+
+ if (flags & I915_WAIT_LOCKED &&
+ __i915_wait_request_check_and_reset(req))
+ continue;
- if (!i915_sw_fence_done(&req->execute)) {
- timeout = __i915_request_wait_for_execute(req, flags, timeout);
- if (timeout < 0)
+ if (signal_pending_state(state, current)) {
+ timeout = -ERESTARTSYS;
goto complete;
+ }
- GEM_BUG_ON(!i915_sw_fence_done(&req->execute));
- }
- GEM_BUG_ON(!i915_sw_fence_done(&req->submit));
- GEM_BUG_ON(!req->global_seqno);
+ if (!timeout) {
+ timeout = -ETIME;
+ goto complete;
+ }
+
+ timeout = io_schedule_timeout(timeout);
+ } while (1);
+
+ GEM_BUG_ON(!intel_wait_has_seqno(&wait));
+ GEM_BUG_ON(!i915_sw_fence_signaled(&req->submit));
/* Optimistic short spin before touching IRQs */
if (i915_spin_request(req, state, 5))
goto complete;
set_current_state(state);
- if (flags & I915_WAIT_LOCKED)
- add_wait_queue(&req->i915->gpu_error.wait_queue, &reset);
-
- intel_wait_init(&wait, req->global_seqno);
if (intel_engine_add_wait(req->engine, &wait))
/* In order to check that we haven't missed the interrupt
* as we enabled it, we need to kick ourselves to do a
*/
goto wakeup;
+ if (flags & I915_WAIT_LOCKED)
+ __i915_wait_request_check_and_reset(req);
+
for (;;) {
if (signal_pending_state(state, current)) {
timeout = -ERESTARTSYS;
timeout = io_schedule_timeout(timeout);
- if (intel_wait_complete(&wait))
+ if (intel_wait_complete(&wait) &&
+ intel_wait_check_request(&wait, req))
break;
set_current_state(state);
* itself, or indirectly by recovering the GPU).
*/
if (flags & I915_WAIT_LOCKED &&
- i915_reset_in_progress(&req->i915->gpu_error)) {
- __set_current_state(TASK_RUNNING);
- i915_reset(req->i915);
- reset_wait_queue(&req->i915->gpu_error.wait_queue,
- &reset);
+ __i915_wait_request_check_and_reset(req))
continue;
- }
/* Only spin if we know the GPU is processing this request */
if (i915_spin_request(req, state, 2))
break;
+
+ if (!intel_wait_check_request(&wait, req)) {
+ intel_engine_remove_wait(req->engine, &wait);
+ goto restart;
+ }
}
intel_engine_remove_wait(req->engine, &wait);
- if (flags & I915_WAIT_LOCKED)
- remove_wait_queue(&req->i915->gpu_error.wait_queue, &reset);
- __set_current_state(TASK_RUNNING);
-
complete:
+ __set_current_state(TASK_RUNNING);
+ if (flags & I915_WAIT_LOCKED)
+ remove_wait_queue(errq, &reset);
+ remove_wait_queue(&req->execute, &exec);
trace_i915_gem_request_wait_end(req);
return timeout;
static void engine_retire_requests(struct intel_engine_cs *engine)
{
struct drm_i915_gem_request *request, *next;
+ u32 seqno = intel_engine_get_seqno(engine);
+ LIST_HEAD(retire);
+ spin_lock_irq(&engine->timeline->lock);
list_for_each_entry_safe(request, next,
&engine->timeline->requests, link) {
- if (!__i915_gem_request_completed(request))
- return;
+ if (!i915_seqno_passed(seqno, request->global_seqno))
+ break;
- i915_gem_request_retire(request);
+ list_move_tail(&request->link, &retire);
}
+ spin_unlock_irq(&engine->timeline->lock);
+
+ list_for_each_entry_safe(request, next, &retire, link)
+ i915_gem_request_retire(request);
}
void i915_gem_retire_requests(struct drm_i915_private *dev_priv)
for_each_engine(engine, dev_priv, id)
engine_retire_requests(engine);
}
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/mock_request.c"
+#include "selftests/i915_gem_request.c"
+#endif
struct drm_file;
struct drm_i915_gem_object;
+struct drm_i915_gem_request;
struct intel_wait {
struct rb_node node;
struct task_struct *tsk;
+ struct drm_i915_gem_request *request;
u32 seqno;
};
* The submit fence is used to await upon all of the request's
* dependencies. When it is signaled, the request is ready to run.
* It is used by the driver to then queue the request for execution.
- *
- * The execute fence is used to signal when the request has been
- * sent to hardware.
- *
- * It is illegal for the submit fence of one request to wait upon the
- * execute fence of an earlier request. It should be sufficient to
- * wait upon the submit fence of the earlier request.
*/
struct i915_sw_fence submit;
- struct i915_sw_fence execute;
wait_queue_t submitq;
- wait_queue_t execq;
+ wait_queue_head_t execute;
/* A list of everyone we wait upon, and everyone who waits upon us.
* Even though we will not be submitted to the hardware before the
struct i915_priotree priotree;
struct i915_dependency dep;
- u32 global_seqno;
-
- /** GEM sequence number associated with the previous request,
- * when the HWS breadcrumb is equal to this the GPU is processing
- * this request.
+ /** GEM sequence number associated with this request on the
+ * global execution timeline. It is zero when the request is not
+ * on the HW queue (i.e. not on the engine timeline list).
+ * Its value is guarded by the timeline spinlock.
*/
- u32 previous_seqno;
+ u32 global_seqno;
/** Position in the ring of the start of the request */
u32 head;
struct drm_i915_file_private *file_priv;
/** file_priv list entry for this request */
- struct list_head client_list;
+ struct list_head client_link;
};
extern const struct dma_fence_ops i915_fence_ops;
struct drm_i915_gem_request * __must_check
i915_gem_request_alloc(struct intel_engine_cs *engine,
struct i915_gem_context *ctx);
-int i915_gem_request_add_to_client(struct drm_i915_gem_request *req,
- struct drm_file *file);
void i915_gem_request_retire_upto(struct drm_i915_gem_request *req);
static inline struct drm_i915_gem_request *
*pdst = src;
}
+/**
+ * i915_gem_request_global_seqno - report the current global seqno
+ * @request - the request
+ *
+ * A request is assigned a global seqno only when it is on the hardware
+ * execution queue. The global seqno can be used to maintain a list of
+ * requests on the same engine in retirement order, for example for
+ * constructing a priority queue for waiting. Prior to its execution, or
+ * if it is subsequently removed in the event of preemption, its global
+ * seqno is zero. As both insertion and removal from the execution queue
+ * may operate in IRQ context, it is not guarded by the usual struct_mutex
+ * BKL. Instead those relying on the global seqno must be prepared for its
+ * value to change between reads. Only when the request is complete can
+ * the global seqno be stable (due to the memory barriers on submitting
+ * the commands to the hardware to write the breadcrumb, if the HWS shows
+ * that it has passed the global seqno and the global seqno is unchanged
+ * after the read, it is indeed complete).
+ */
+static u32
+i915_gem_request_global_seqno(const struct drm_i915_gem_request *request)
+{
+ return READ_ONCE(request->global_seqno);
+}
+
int
i915_gem_request_await_object(struct drm_i915_gem_request *to,
struct drm_i915_gem_object *obj,
void __i915_add_request(struct drm_i915_gem_request *req, bool flush_caches);
#define i915_add_request(req) \
- __i915_add_request(req, true)
-#define i915_add_request_no_flush(req) \
__i915_add_request(req, false)
void __i915_gem_request_submit(struct drm_i915_gem_request *request);
void i915_gem_request_submit(struct drm_i915_gem_request *request);
+void __i915_gem_request_unsubmit(struct drm_i915_gem_request *request);
+void i915_gem_request_unsubmit(struct drm_i915_gem_request *request);
+
struct intel_rps_client;
#define NO_WAITBOOST ERR_PTR(-1)
#define IS_RPS_CLIENT(p) (!IS_ERR(p))
}
static inline bool
-__i915_gem_request_started(const struct drm_i915_gem_request *req)
+__i915_gem_request_started(const struct drm_i915_gem_request *req, u32 seqno)
{
- GEM_BUG_ON(!req->global_seqno);
+ GEM_BUG_ON(!seqno);
return i915_seqno_passed(intel_engine_get_seqno(req->engine),
- req->previous_seqno);
+ seqno - 1);
}
static inline bool
i915_gem_request_started(const struct drm_i915_gem_request *req)
{
- if (!req->global_seqno)
+ u32 seqno;
+
+ seqno = i915_gem_request_global_seqno(req);
+ if (!seqno)
return false;
- return __i915_gem_request_started(req);
+ return __i915_gem_request_started(req, seqno);
}
static inline bool
-__i915_gem_request_completed(const struct drm_i915_gem_request *req)
+__i915_gem_request_completed(const struct drm_i915_gem_request *req, u32 seqno)
{
- GEM_BUG_ON(!req->global_seqno);
- return i915_seqno_passed(intel_engine_get_seqno(req->engine),
- req->global_seqno);
+ GEM_BUG_ON(!seqno);
+ return i915_seqno_passed(intel_engine_get_seqno(req->engine), seqno) &&
+ seqno == i915_gem_request_global_seqno(req);
}
static inline bool
i915_gem_request_completed(const struct drm_i915_gem_request *req)
{
- if (!req->global_seqno)
+ u32 seqno;
+
+ seqno = i915_gem_request_global_seqno(req);
+ if (!seqno)
return false;
- return __i915_gem_request_completed(req);
+ return __i915_gem_request_completed(req, seqno);
}
bool __i915_spin_request(const struct drm_i915_gem_request *request,
- int state, unsigned long timeout_us);
+ u32 seqno, int state, unsigned long timeout_us);
static inline bool i915_spin_request(const struct drm_i915_gem_request *request,
int state, unsigned long timeout_us)
{
- return (__i915_gem_request_started(request) &&
- __i915_spin_request(request, state, timeout_us));
+ u32 seqno;
+
+ seqno = i915_gem_request_global_seqno(request);
+ if (!seqno)
+ return 0;
+
+ return (__i915_gem_request_started(request, seqno) &&
+ __i915_spin_request(request, seqno, state, timeout_us));
}
/* We treat requests as fences. This is not be to confused with our
if (!(flags & I915_SHRINK_ACTIVE) &&
(i915_gem_object_is_active(obj) ||
- obj->framebuffer_references))
+ i915_gem_object_is_framebuffer(obj)))
continue;
if (!can_release_pages(obj))
{
unsigned long freed;
+ intel_runtime_pm_get(dev_priv);
freed = i915_gem_shrink(dev_priv, -1UL,
I915_SHRINK_BOUND |
I915_SHRINK_UNBOUND |
I915_SHRINK_ACTIVE);
+ intel_runtime_pm_put(dev_priv);
+
synchronize_rcu(); /* wait for our earlier RCU delayed slab frees */
return freed;
if (!i915_gem_shrinker_lock_uninterruptible(dev_priv, &slu, 5000))
return NOTIFY_DONE;
- intel_runtime_pm_get(dev_priv);
freed_pages = i915_gem_shrink_all(dev_priv);
- intel_runtime_pm_put(dev_priv);
/* Because we may be allocating inside our own driver, we cannot
* assert that there are no objects with pinned pages that are not
mutex_unlock(&dev_priv->mm.stolen_lock);
}
-static unsigned long i915_stolen_to_physical(struct drm_i915_private *dev_priv)
+static dma_addr_t i915_stolen_to_dma(struct drm_i915_private *dev_priv)
{
struct pci_dev *pdev = dev_priv->drm.pdev;
struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct resource *r;
- u32 base;
+ dma_addr_t base;
/* Almost universally we can find the Graphics Base of Stolen Memory
* at register BSM (0x5c) in the igfx configuration space. On a few
base = tom - tseg_size - ggtt->stolen_size;
}
- if (base == 0)
+ if (base == 0 || add_overflows(base, ggtt->stolen_size))
return 0;
/* make sure we don't clobber the GTT if it's within stolen memory */
if (INTEL_GEN(dev_priv) <= 4 &&
!IS_G33(dev_priv) && !IS_PINEVIEW(dev_priv) && !IS_G4X(dev_priv)) {
struct {
- u32 start, end;
+ dma_addr_t start, end;
} stolen[2] = {
{ .start = base, .end = base + ggtt->stolen_size, },
{ .start = base, .end = base + ggtt->stolen_size, },
if (stolen[0].start != stolen[1].start ||
stolen[0].end != stolen[1].end) {
+ dma_addr_t end = base + ggtt->stolen_size - 1;
+
DRM_DEBUG_KMS("GTT within stolen memory at 0x%llx-0x%llx\n",
(unsigned long long)ggtt_start,
(unsigned long long)ggtt_end - 1);
- DRM_DEBUG_KMS("Stolen memory adjusted to 0x%x-0x%x\n",
- base, base + (u32)ggtt->stolen_size - 1);
+ DRM_DEBUG_KMS("Stolen memory adjusted to %pad - %pad\n",
+ &base, &end);
}
}
* range. Apparently this works.
*/
if (r == NULL && !IS_GEN3(dev_priv)) {
- DRM_ERROR("conflict detected with stolen region: [0x%08x - 0x%08x]\n",
- base, base + (uint32_t)ggtt->stolen_size);
+ dma_addr_t end = base + ggtt->stolen_size;
+
+ DRM_ERROR("conflict detected with stolen region: [%pad - %pad]\n",
+ &base, &end);
base = 0;
}
}
}
static void g4x_get_stolen_reserved(struct drm_i915_private *dev_priv,
- phys_addr_t *base, u32 *size)
+ dma_addr_t *base, u32 *size)
{
struct i915_ggtt *ggtt = &dev_priv->ggtt;
uint32_t reg_val = I915_READ(IS_GM45(dev_priv) ?
CTG_STOLEN_RESERVED :
ELK_STOLEN_RESERVED);
- phys_addr_t stolen_top = dev_priv->mm.stolen_base + ggtt->stolen_size;
+ dma_addr_t stolen_top = dev_priv->mm.stolen_base + ggtt->stolen_size;
*base = (reg_val & G4X_STOLEN_RESERVED_ADDR2_MASK) << 16;
}
static void gen6_get_stolen_reserved(struct drm_i915_private *dev_priv,
- phys_addr_t *base, u32 *size)
+ dma_addr_t *base, u32 *size)
{
uint32_t reg_val = I915_READ(GEN6_STOLEN_RESERVED);
}
static void gen7_get_stolen_reserved(struct drm_i915_private *dev_priv,
- phys_addr_t *base, u32 *size)
+ dma_addr_t *base, u32 *size)
{
uint32_t reg_val = I915_READ(GEN6_STOLEN_RESERVED);
}
static void chv_get_stolen_reserved(struct drm_i915_private *dev_priv,
- phys_addr_t *base, u32 *size)
+ dma_addr_t *base, u32 *size)
{
uint32_t reg_val = I915_READ(GEN6_STOLEN_RESERVED);
}
static void bdw_get_stolen_reserved(struct drm_i915_private *dev_priv,
- phys_addr_t *base, u32 *size)
+ dma_addr_t *base, u32 *size)
{
struct i915_ggtt *ggtt = &dev_priv->ggtt;
uint32_t reg_val = I915_READ(GEN6_STOLEN_RESERVED);
- phys_addr_t stolen_top;
+ dma_addr_t stolen_top;
stolen_top = dev_priv->mm.stolen_base + ggtt->stolen_size;
int i915_gem_init_stolen(struct drm_i915_private *dev_priv)
{
struct i915_ggtt *ggtt = &dev_priv->ggtt;
- phys_addr_t reserved_base, stolen_top;
+ dma_addr_t reserved_base, stolen_top;
u32 reserved_total, reserved_size;
u32 stolen_usable_start;
if (ggtt->stolen_size == 0)
return 0;
- dev_priv->mm.stolen_base = i915_stolen_to_physical(dev_priv);
+ dev_priv->mm.stolen_base = i915_stolen_to_dma(dev_priv);
if (dev_priv->mm.stolen_base == 0)
return 0;
if (reserved_base < dev_priv->mm.stolen_base ||
reserved_base + reserved_size > stolen_top) {
- phys_addr_t reserved_top = reserved_base + reserved_size;
- DRM_DEBUG_KMS("Stolen reserved area [%pa - %pa] outside stolen memory [%pa - %pa]\n",
+ dma_addr_t reserved_top = reserved_base + reserved_size;
+ DRM_DEBUG_KMS("Stolen reserved area [%pad - %pad] outside stolen memory [%pad - %pad]\n",
&reserved_base, &reserved_top,
&dev_priv->mm.stolen_base, &stolen_top);
return 0;
if (stride > 8192)
return false;
- if (IS_GEN3(i915)) {
- if (obj->base.size > I830_FENCE_MAX_SIZE_VAL << 20)
- return false;
- } else {
- if (obj->base.size > I830_FENCE_MAX_SIZE_VAL << 19)
- return false;
- }
+ if (!is_power_of_2(stride))
+ return false;
}
if (IS_GEN2(i915) ||
if (!stride || !IS_ALIGNED(stride, tile_width))
return false;
- /* 965+ just needs multiples of tile width */
- if (INTEL_GEN(i915) >= 4)
- return true;
-
- /* Pre-965 needs power of two tile widths */
- return is_power_of_2(stride);
+ return true;
}
static bool i915_vma_fence_prepare(struct i915_vma *vma,
if ((tiling | stride) == obj->tiling_and_stride)
return 0;
- if (obj->framebuffer_references)
+ if (i915_gem_object_is_framebuffer(obj))
return -EBUSY;
/* We need to rebind the object if its current allocation
if (err)
return err;
+ i915_gem_object_lock(obj);
+ if (i915_gem_object_is_framebuffer(obj)) {
+ i915_gem_object_unlock(obj);
+ return -EBUSY;
+ }
+
/* If the memory has unknown (i.e. varying) swizzling, we pin the
* pages to prevent them being swapped out and causing corruption
* due to the change in swizzling.
}
obj->tiling_and_stride = tiling | stride;
+ i915_gem_object_unlock(obj);
/* Force the fence to be reacquired for GTT access */
i915_gem_release_mmap(obj);
struct intel_timeline {
u64 fence_context;
- u32 last_submitted_seqno;
+ u32 seqno;
+
+ /**
+ * Count of outstanding requests, from the time they are constructed
+ * to the moment they are retired. Loosely coupled to hardware.
+ */
+ u32 inflight_seqnos;
spinlock_t lock;
struct i915_gem_timeline {
struct list_head link;
- atomic_t seqno;
struct drm_i915_private *i915;
const char *name;
{
struct i915_mmu_object *mo = container_of(work, typeof(*mo), work);
struct drm_i915_gem_object *obj = mo->obj;
- struct drm_device *dev = obj->base.dev;
+ struct work_struct *active;
+
+ /* Cancel any active worker and force us to re-evaluate gup */
+ mutex_lock(&obj->mm.lock);
+ active = fetch_and_zero(&obj->userptr.work);
+ mutex_unlock(&obj->mm.lock);
+ if (active)
+ goto out;
i915_gem_object_wait(obj, I915_WAIT_ALL, MAX_SCHEDULE_TIMEOUT, NULL);
- mutex_lock(&dev->struct_mutex);
- /* Cancel any active worker and force us to re-evaluate gup */
- obj->userptr.work = NULL;
+ mutex_lock(&obj->base.dev->struct_mutex);
/* We are inside a kthread context and can't be interrupted */
if (i915_gem_object_unbind(obj) == 0)
atomic_read(&obj->mm.pages_pin_count),
obj->pin_display);
+ mutex_unlock(&obj->base.dev->struct_mutex);
+
+out:
i915_gem_object_put(obj);
- mutex_unlock(&dev->struct_mutex);
}
static void add_object(struct i915_mmu_object *mo)
del_object(mo);
spin_unlock(&mn->lock);
- flush_workqueue(mn->wq);
+ if (!list_empty(&cancelled))
+ flush_workqueue(mn->wq);
}
static const struct mmu_notifier_ops i915_gem_userptr_notifier = {
}
obj->userptr.work = ERR_CAST(pages);
+ if (IS_ERR(pages))
+ __i915_gem_userptr_set_active(obj, false);
}
mutex_unlock(&obj->mm.lock);
}
static struct sg_table *
-__i915_gem_userptr_get_pages_schedule(struct drm_i915_gem_object *obj,
- bool *active)
+__i915_gem_userptr_get_pages_schedule(struct drm_i915_gem_object *obj)
{
struct get_pages_work *work;
INIT_WORK(&work->work, __i915_gem_userptr_get_pages_worker);
schedule_work(&work->work);
- *active = true;
return ERR_PTR(-EAGAIN);
}
i915_gem_userptr_get_pages(struct drm_i915_gem_object *obj)
{
const int num_pages = obj->base.size >> PAGE_SHIFT;
+ struct mm_struct *mm = obj->userptr.mm->mm;
struct page **pvec;
struct sg_table *pages;
- int pinned, ret;
bool active;
+ int pinned;
/* If userspace should engineer that these pages are replaced in
* the vma between us binding this page into the GTT and completion
return ERR_PTR(-EAGAIN);
}
- /* Let the mmu-notifier know that we have begun and need cancellation */
- ret = __i915_gem_userptr_set_active(obj, true);
- if (ret)
- return ERR_PTR(ret);
-
pvec = NULL;
pinned = 0;
- if (obj->userptr.mm->mm == current->mm) {
- pvec = drm_malloc_gfp(num_pages, sizeof(struct page *),
- GFP_TEMPORARY);
- if (pvec == NULL) {
- __i915_gem_userptr_set_active(obj, false);
- return ERR_PTR(-ENOMEM);
- }
- pinned = __get_user_pages_fast(obj->userptr.ptr, num_pages,
- !obj->userptr.read_only, pvec);
+ if (mm == current->mm) {
+ pvec = drm_malloc_gfp(num_pages, sizeof(struct page *),
+ GFP_TEMPORARY |
+ __GFP_NORETRY |
+ __GFP_NOWARN);
+ if (pvec) /* defer to worker if malloc fails */
+ pinned = __get_user_pages_fast(obj->userptr.ptr,
+ num_pages,
+ !obj->userptr.read_only,
+ pvec);
}
active = false;
- if (pinned < 0)
- pages = ERR_PTR(pinned), pinned = 0;
- else if (pinned < num_pages)
- pages = __i915_gem_userptr_get_pages_schedule(obj, &active);
- else
+ if (pinned < 0) {
+ pages = ERR_PTR(pinned);
+ pinned = 0;
+ } else if (pinned < num_pages) {
+ pages = __i915_gem_userptr_get_pages_schedule(obj);
+ active = pages == ERR_PTR(-EAGAIN);
+ } else {
pages = __i915_gem_userptr_set_pages(obj, pvec, num_pages);
- if (IS_ERR(pages)) {
- __i915_gem_userptr_set_active(obj, active);
- release_pages(pvec, pinned, 0);
+ active = !IS_ERR(pages);
}
+ if (active)
+ __i915_gem_userptr_set_active(obj, true);
+
+ if (IS_ERR(pages))
+ release_pages(pvec, pinned, 0);
drm_free_large(pvec);
+
return pages;
}
}
static void error_print_instdone(struct drm_i915_error_state_buf *m,
- struct drm_i915_error_engine *ee)
+ const struct drm_i915_error_engine *ee)
{
int slice;
int subslice;
static void error_print_request(struct drm_i915_error_state_buf *m,
const char *prefix,
- struct drm_i915_error_request *erq)
+ const struct drm_i915_error_request *erq)
{
if (!erq->seqno)
return;
erq->head, erq->tail);
}
+static void error_print_context(struct drm_i915_error_state_buf *m,
+ const char *header,
+ const struct drm_i915_error_context *ctx)
+{
+ err_printf(m, "%s%s[%d] user_handle %d hw_id %d, ban score %d guilty %d active %d\n",
+ header, ctx->comm, ctx->pid, ctx->handle, ctx->hw_id,
+ ctx->ban_score, ctx->guilty, ctx->active);
+}
+
static void error_print_engine(struct drm_i915_error_state_buf *m,
- struct drm_i915_error_engine *ee)
+ const struct drm_i915_error_engine *ee)
{
err_printf(m, "%s command stream:\n", engine_str(ee->engine_id));
err_printf(m, " START: 0x%08x\n", ee->start);
error_print_request(m, " ELSP[0]: ", &ee->execlist[0]);
error_print_request(m, " ELSP[1]: ", &ee->execlist[1]);
+ error_print_context(m, " Active context: ", &ee->context);
}
void i915_error_printf(struct drm_i915_error_state_buf *e, const char *f, ...)
#undef PRINT_FLAG
}
+static __always_inline void err_print_param(struct drm_i915_error_state_buf *m,
+ const char *name,
+ const char *type,
+ const void *x)
+{
+ if (!__builtin_strcmp(type, "bool"))
+ err_printf(m, "i915.%s=%s\n", name, yesno(*(const bool *)x));
+ else if (!__builtin_strcmp(type, "int"))
+ err_printf(m, "i915.%s=%d\n", name, *(const int *)x);
+ else if (!__builtin_strcmp(type, "unsigned int"))
+ err_printf(m, "i915.%s=%u\n", name, *(const unsigned int *)x);
+ else if (!__builtin_strcmp(type, "char *"))
+ err_printf(m, "i915.%s=%s\n", name, *(const char **)x);
+ else
+ BUILD_BUG();
+}
+
+static void err_print_params(struct drm_i915_error_state_buf *m,
+ const struct i915_params *p)
+{
+#define PRINT(T, x) err_print_param(m, #x, #T, &p->x);
+ I915_PARAMS_FOR_EACH(PRINT);
+#undef PRINT
+}
+
+static void err_print_pciid(struct drm_i915_error_state_buf *m,
+ struct drm_i915_private *i915)
+{
+ struct pci_dev *pdev = i915->drm.pdev;
+
+ err_printf(m, "PCI ID: 0x%04x\n", pdev->device);
+ err_printf(m, "PCI Revision: 0x%02x\n", pdev->revision);
+ err_printf(m, "PCI Subsystem: %04x:%04x\n",
+ pdev->subsystem_vendor,
+ pdev->subsystem_device);
+}
+
int i915_error_state_to_str(struct drm_i915_error_state_buf *m,
- const struct i915_error_state_file_priv *error_priv)
+ const struct i915_gpu_state *error)
{
- struct drm_i915_private *dev_priv = error_priv->i915;
- struct pci_dev *pdev = dev_priv->drm.pdev;
- struct drm_i915_error_state *error = error_priv->error;
+ struct drm_i915_private *dev_priv = m->i915;
struct drm_i915_error_object *obj;
int i, j;
if (!error) {
- err_printf(m, "no error state collected\n");
- goto out;
+ err_printf(m, "No error state collected\n");
+ return 0;
}
- err_printf(m, "%s\n", error->error_msg);
+ if (*error->error_msg)
+ err_printf(m, "%s\n", error->error_msg);
err_printf(m, "Kernel: " UTS_RELEASE "\n");
err_printf(m, "Time: %ld s %ld us\n",
error->time.tv_sec, error->time.tv_usec);
error->boottime.tv_sec, error->boottime.tv_usec);
err_printf(m, "Uptime: %ld s %ld us\n",
error->uptime.tv_sec, error->uptime.tv_usec);
- err_print_capabilities(m, &error->device_info);
for (i = 0; i < ARRAY_SIZE(error->engine); i++) {
if (error->engine[i].hangcheck_stalled &&
- error->engine[i].pid != -1) {
- err_printf(m, "Active process (on ring %s): %s [%d], context bans %d\n",
+ error->engine[i].context.pid) {
+ err_printf(m, "Active process (on ring %s): %s [%d], score %d\n",
engine_str(i),
- error->engine[i].comm,
- error->engine[i].pid,
- error->engine[i].context_bans);
+ error->engine[i].context.comm,
+ error->engine[i].context.pid,
+ error->engine[i].context.ban_score);
}
}
err_printf(m, "Reset count: %u\n", error->reset_count);
err_printf(m, "Suspend count: %u\n", error->suspend_count);
err_printf(m, "Platform: %s\n", intel_platform_name(error->device_info.platform));
- err_printf(m, "PCI ID: 0x%04x\n", pdev->device);
- err_printf(m, "PCI Revision: 0x%02x\n", pdev->revision);
- err_printf(m, "PCI Subsystem: %04x:%04x\n",
- pdev->subsystem_vendor,
- pdev->subsystem_device);
+ err_print_pciid(m, error->i915);
+
err_printf(m, "IOMMU enabled?: %d\n", error->iommu);
if (HAS_CSR(dev_priv)) {
CSR_VERSION_MINOR(csr->version));
}
+ err_printf(m, "GT awake: %s\n", yesno(error->awake));
+ err_printf(m, "RPM wakelock: %s\n", yesno(error->wakelock));
+ err_printf(m, "PM suspended: %s\n", yesno(error->suspended));
err_printf(m, "EIR: 0x%08x\n", error->eir);
err_printf(m, "IER: 0x%08x\n", error->ier);
- if (INTEL_GEN(dev_priv) >= 8) {
- for (i = 0; i < 4; i++)
- err_printf(m, "GTIER gt %d: 0x%08x\n", i,
- error->gtier[i]);
- } else if (HAS_PCH_SPLIT(dev_priv) || IS_VALLEYVIEW(dev_priv))
- err_printf(m, "GTIER: 0x%08x\n", error->gtier[0]);
+ for (i = 0; i < error->ngtier; i++)
+ err_printf(m, "GTIER[%d]: 0x%08x\n", i, error->gtier[i]);
err_printf(m, "PGTBL_ER: 0x%08x\n", error->pgtbl_er);
err_printf(m, "FORCEWAKE: 0x%08x\n", error->forcewake);
err_printf(m, "DERRMR: 0x%08x\n", error->derrmr);
err_printf(m, "CCID: 0x%08x\n", error->ccid);
err_printf(m, "Missed interrupts: 0x%08lx\n", dev_priv->gpu_error.missed_irq_rings);
- for (i = 0; i < dev_priv->num_fence_regs; i++)
+ for (i = 0; i < error->nfence; i++)
err_printf(m, " fence[%d] = %08llx\n", i, error->fence[i]);
if (INTEL_GEN(dev_priv) >= 6) {
error->pinned_bo_count);
for (i = 0; i < ARRAY_SIZE(error->engine); i++) {
- struct drm_i915_error_engine *ee = &error->engine[i];
+ const struct drm_i915_error_engine *ee = &error->engine[i];
obj = ee->batchbuffer;
if (obj) {
err_puts(m, dev_priv->engine[i]->name);
- if (ee->pid != -1)
- err_printf(m, " (submitted by %s [%d], bans %d)",
- ee->comm,
- ee->pid,
- ee->context_bans);
+ if (ee->context.pid)
+ err_printf(m, " (submitted by %s [%d], ctx %d [%d], score %d)",
+ ee->context.comm,
+ ee->context.pid,
+ ee->context.handle,
+ ee->context.hw_id,
+ ee->context.ban_score);
err_printf(m, " --- gtt_offset = 0x%08x %08x\n",
upper_32_bits(obj->gtt_offset),
lower_32_bits(obj->gtt_offset));
intel_overlay_print_error_state(m, error->overlay);
if (error->display)
- intel_display_print_error_state(m, dev_priv, error->display);
+ intel_display_print_error_state(m, error->display);
+
+ err_print_capabilities(m, &error->device_info);
+ err_print_params(m, &error->params);
-out:
if (m->bytes == 0 && m->err)
return m->err;
kfree(obj);
}
-static void i915_error_state_free(struct kref *error_ref)
+static __always_inline void free_param(const char *type, void *x)
+{
+ if (!__builtin_strcmp(type, "char *"))
+ kfree(*(void **)x);
+}
+
+void __i915_gpu_state_free(struct kref *error_ref)
{
- struct drm_i915_error_state *error = container_of(error_ref,
- typeof(*error), ref);
+ struct i915_gpu_state *error =
+ container_of(error_ref, typeof(*error), ref);
int i;
for (i = 0; i < ARRAY_SIZE(error->engine); i++) {
kfree(error->overlay);
kfree(error->display);
+
+#define FREE(T, x) free_param(#T, &error->params.x);
+ I915_PARAMS_FOR_EACH(FREE);
+#undef FREE
+
kfree(error);
}
* It's only a small step better than a random number in its current form.
*/
static uint32_t i915_error_generate_code(struct drm_i915_private *dev_priv,
- struct drm_i915_error_state *error,
+ struct i915_gpu_state *error,
int *engine_id)
{
uint32_t error_code = 0;
}
static void i915_gem_record_fences(struct drm_i915_private *dev_priv,
- struct drm_i915_error_state *error)
+ struct i915_gpu_state *error)
{
int i;
- if (IS_GEN3(dev_priv) || IS_GEN2(dev_priv)) {
+ if (INTEL_GEN(dev_priv) >= 6) {
for (i = 0; i < dev_priv->num_fence_regs; i++)
- error->fence[i] = I915_READ(FENCE_REG(i));
- } else if (IS_GEN5(dev_priv) || IS_GEN4(dev_priv)) {
+ error->fence[i] = I915_READ64(FENCE_REG_GEN6_LO(i));
+ } else if (INTEL_GEN(dev_priv) >= 4) {
for (i = 0; i < dev_priv->num_fence_regs; i++)
error->fence[i] = I915_READ64(FENCE_REG_965_LO(i));
- } else if (INTEL_GEN(dev_priv) >= 6) {
+ } else {
for (i = 0; i < dev_priv->num_fence_regs; i++)
- error->fence[i] = I915_READ64(FENCE_REG_GEN6_LO(i));
+ error->fence[i] = I915_READ(FENCE_REG(i));
}
+ error->nfence = i;
}
static inline u32
return idx;
}
-static void gen8_record_semaphore_state(struct drm_i915_error_state *error,
+static void gen8_record_semaphore_state(struct i915_gpu_state *error,
struct intel_engine_cs *engine,
struct drm_i915_error_engine *ee)
{
if (RB_EMPTY_ROOT(&b->waiters))
return;
- if (!spin_trylock_irq(&b->lock)) {
+ if (!spin_trylock_irq(&b->rb_lock)) {
ee->waiters = ERR_PTR(-EDEADLK);
return;
}
count = 0;
for (rb = rb_first(&b->waiters); rb != NULL; rb = rb_next(rb))
count++;
- spin_unlock_irq(&b->lock);
+ spin_unlock_irq(&b->rb_lock);
waiter = NULL;
if (count)
if (!waiter)
return;
- if (!spin_trylock_irq(&b->lock)) {
+ if (!spin_trylock_irq(&b->rb_lock)) {
kfree(waiter);
ee->waiters = ERR_PTR(-EDEADLK);
return;
ee->waiters = waiter;
for (rb = rb_first(&b->waiters); rb; rb = rb_next(rb)) {
- struct intel_wait *w = container_of(rb, typeof(*w), node);
+ struct intel_wait *w = rb_entry(rb, typeof(*w), node);
strcpy(waiter->comm, w->tsk->comm);
waiter->pid = w->tsk->pid;
if (++ee->num_waiters == count)
break;
}
- spin_unlock_irq(&b->lock);
+ spin_unlock_irq(&b->rb_lock);
}
-static void error_record_engine_registers(struct drm_i915_error_state *error,
+static void error_record_engine_registers(struct i915_gpu_state *error,
struct intel_engine_cs *engine,
struct drm_i915_error_engine *ee)
{
&ee->execlist[n]);
}
+static void record_context(struct drm_i915_error_context *e,
+ struct i915_gem_context *ctx)
+{
+ if (ctx->pid) {
+ struct task_struct *task;
+
+ rcu_read_lock();
+ task = pid_task(ctx->pid, PIDTYPE_PID);
+ if (task) {
+ strcpy(e->comm, task->comm);
+ e->pid = task->pid;
+ }
+ rcu_read_unlock();
+ }
+
+ e->handle = ctx->user_handle;
+ e->hw_id = ctx->hw_id;
+ e->ban_score = ctx->ban_score;
+ e->guilty = ctx->guilty_count;
+ e->active = ctx->active_count;
+}
+
static void i915_gem_record_rings(struct drm_i915_private *dev_priv,
- struct drm_i915_error_state *error)
+ struct i915_gpu_state *error)
{
struct i915_ggtt *ggtt = &dev_priv->ggtt;
int i;
struct drm_i915_error_engine *ee = &error->engine[i];
struct drm_i915_gem_request *request;
- ee->pid = -1;
ee->engine_id = -1;
if (!engine)
request = i915_gem_find_active_request(engine);
if (request) {
struct intel_ring *ring;
- struct pid *pid;
ee->vm = request->ctx->ppgtt ?
&request->ctx->ppgtt->base : &ggtt->base;
+ record_context(&ee->context, request->ctx);
+
/* We need to copy these to an anonymous buffer
* as the simplest method to avoid being overwritten
* by userspace.
i915_error_object_create(dev_priv,
request->ctx->engine[i].state);
- pid = request->ctx->pid;
- if (pid) {
- struct task_struct *task;
-
- rcu_read_lock();
- task = pid_task(pid, PIDTYPE_PID);
- if (task) {
- strcpy(ee->comm, task->comm);
- ee->pid = task->pid;
- }
- rcu_read_unlock();
- }
-
error->simulated |=
i915_gem_context_no_error_capture(request->ctx);
}
static void i915_gem_capture_vm(struct drm_i915_private *dev_priv,
- struct drm_i915_error_state *error,
+ struct i915_gpu_state *error,
struct i915_address_space *vm,
int idx)
{
}
static void i915_capture_active_buffers(struct drm_i915_private *dev_priv,
- struct drm_i915_error_state *error)
+ struct i915_gpu_state *error)
{
int cnt = 0, i, j;
}
static void i915_capture_pinned_buffers(struct drm_i915_private *dev_priv,
- struct drm_i915_error_state *error)
+ struct i915_gpu_state *error)
{
struct i915_address_space *vm = &dev_priv->ggtt.base;
struct drm_i915_error_buffer *bo;
}
static void i915_gem_capture_guc_log_buffer(struct drm_i915_private *dev_priv,
- struct drm_i915_error_state *error)
+ struct i915_gpu_state *error)
{
/* Capturing log buf contents won't be useful if logging was disabled */
if (!dev_priv->guc.log.vma || (i915.guc_log_level < 0))
/* Capture all registers which don't fit into another category. */
static void i915_capture_reg_state(struct drm_i915_private *dev_priv,
- struct drm_i915_error_state *error)
+ struct i915_gpu_state *error)
{
int i;
error->ier = I915_READ(GEN8_DE_MISC_IER);
for (i = 0; i < 4; i++)
error->gtier[i] = I915_READ(GEN8_GT_IER(i));
+ error->ngtier = 4;
} else if (HAS_PCH_SPLIT(dev_priv)) {
error->ier = I915_READ(DEIER);
error->gtier[0] = I915_READ(GTIER);
+ error->ngtier = 1;
} else if (IS_GEN2(dev_priv)) {
error->ier = I915_READ16(IER);
} else if (!IS_VALLEYVIEW(dev_priv)) {
}
static void i915_error_capture_msg(struct drm_i915_private *dev_priv,
- struct drm_i915_error_state *error,
+ struct i915_gpu_state *error,
u32 engine_mask,
const char *error_msg)
{
"GPU HANG: ecode %d:%d:0x%08x",
INTEL_GEN(dev_priv), engine_id, ecode);
- if (engine_id != -1 && error->engine[engine_id].pid != -1)
+ if (engine_id != -1 && error->engine[engine_id].context.pid)
len += scnprintf(error->error_msg + len,
sizeof(error->error_msg) - len,
", in %s [%d]",
- error->engine[engine_id].comm,
- error->engine[engine_id].pid);
+ error->engine[engine_id].context.comm,
+ error->engine[engine_id].context.pid);
scnprintf(error->error_msg + len, sizeof(error->error_msg) - len,
", reason: %s, action: %s",
}
static void i915_capture_gen_state(struct drm_i915_private *dev_priv,
- struct drm_i915_error_state *error)
+ struct i915_gpu_state *error)
{
+ error->awake = dev_priv->gt.awake;
+ error->wakelock = atomic_read(&dev_priv->pm.wakeref_count);
+ error->suspended = dev_priv->pm.suspended;
+
error->iommu = -1;
#ifdef CONFIG_INTEL_IOMMU
error->iommu = intel_iommu_gfx_mapped;
sizeof(error->device_info));
}
+static __always_inline void dup_param(const char *type, void *x)
+{
+ if (!__builtin_strcmp(type, "char *"))
+ *(void **)x = kstrdup(*(void **)x, GFP_ATOMIC);
+}
+
static int capture(void *data)
{
- struct drm_i915_error_state *error = data;
+ struct i915_gpu_state *error = data;
+
+ do_gettimeofday(&error->time);
+ error->boottime = ktime_to_timeval(ktime_get_boottime());
+ error->uptime =
+ ktime_to_timeval(ktime_sub(ktime_get(),
+ error->i915->gt.last_init_time));
+
+ error->params = i915;
+#define DUP(T, x) dup_param(#T, &error->params.x);
+ I915_PARAMS_FOR_EACH(DUP);
+#undef DUP
i915_capture_gen_state(error->i915, error);
i915_capture_reg_state(error->i915, error);
i915_capture_pinned_buffers(error->i915, error);
i915_gem_capture_guc_log_buffer(error->i915, error);
- do_gettimeofday(&error->time);
- error->boottime = ktime_to_timeval(ktime_get_boottime());
- error->uptime =
- ktime_to_timeval(ktime_sub(ktime_get(),
- error->i915->gt.last_init_time));
-
error->overlay = intel_overlay_capture_error_state(error->i915);
error->display = intel_display_capture_error_state(error->i915);
#define DAY_AS_SECONDS(x) (24 * 60 * 60 * (x))
+struct i915_gpu_state *
+i915_capture_gpu_state(struct drm_i915_private *i915)
+{
+ struct i915_gpu_state *error;
+
+ error = kzalloc(sizeof(*error), GFP_ATOMIC);
+ if (!error)
+ return NULL;
+
+ kref_init(&error->ref);
+ error->i915 = i915;
+
+ stop_machine(capture, error, NULL);
+
+ return error;
+}
+
/**
* i915_capture_error_state - capture an error record for later analysis
* @dev: drm device
const char *error_msg)
{
static bool warned;
- struct drm_i915_error_state *error;
+ struct i915_gpu_state *error;
unsigned long flags;
if (!i915.error_capture)
if (READ_ONCE(dev_priv->gpu_error.first_error))
return;
- /* Account for pipe specific data like PIPE*STAT */
- error = kzalloc(sizeof(*error), GFP_ATOMIC);
+ error = i915_capture_gpu_state(dev_priv);
if (!error) {
DRM_DEBUG_DRIVER("out of memory, not capturing error state\n");
return;
}
- kref_init(&error->ref);
- error->i915 = dev_priv;
-
- stop_machine(capture, error, NULL);
-
i915_error_capture_msg(dev_priv, error, engine_mask, error_msg);
DRM_INFO("%s\n", error->error_msg);
}
if (error) {
- i915_error_state_free(&error->ref);
+ __i915_gpu_state_free(&error->ref);
return;
}
}
}
-void i915_error_state_get(struct drm_device *dev,
- struct i915_error_state_file_priv *error_priv)
+struct i915_gpu_state *
+i915_first_error_state(struct drm_i915_private *i915)
{
- struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_gpu_state *error;
- spin_lock_irq(&dev_priv->gpu_error.lock);
- error_priv->error = dev_priv->gpu_error.first_error;
- if (error_priv->error)
- kref_get(&error_priv->error->ref);
- spin_unlock_irq(&dev_priv->gpu_error.lock);
-}
+ spin_lock_irq(&i915->gpu_error.lock);
+ error = i915->gpu_error.first_error;
+ if (error)
+ i915_gpu_state_get(error);
+ spin_unlock_irq(&i915->gpu_error.lock);
-void i915_error_state_put(struct i915_error_state_file_priv *error_priv)
-{
- if (error_priv->error)
- kref_put(&error_priv->error->ref, i915_error_state_free);
+ return error;
}
-void i915_destroy_error_state(struct drm_i915_private *dev_priv)
+void i915_reset_error_state(struct drm_i915_private *i915)
{
- struct drm_i915_error_state *error;
+ struct i915_gpu_state *error;
- spin_lock_irq(&dev_priv->gpu_error.lock);
- error = dev_priv->gpu_error.first_error;
- dev_priv->gpu_error.first_error = NULL;
- spin_unlock_irq(&dev_priv->gpu_error.lock);
+ spin_lock_irq(&i915->gpu_error.lock);
+ error = i915->gpu_error.first_error;
+ i915->gpu_error.first_error = NULL;
+ spin_unlock_irq(&i915->gpu_error.lock);
- if (error)
- kref_put(&error->ref, i915_error_state_free);
+ i915_gpu_state_put(error);
}
#include "i915_drv.h"
#include "intel_uc.h"
+#include <trace/events/dma_fence.h>
+
/**
* DOC: GuC-based command submission
*
u32 freespace;
int ret;
- spin_lock(&client->wq_lock);
+ spin_lock_irq(&client->wq_lock);
freespace = CIRC_SPACE(client->wq_tail, desc->head, client->wq_size);
freespace -= client->wq_rsvd;
if (likely(freespace >= wqi_size)) {
client->no_wq_space++;
ret = -EAGAIN;
}
- spin_unlock(&client->wq_lock);
+ spin_unlock_irq(&client->wq_lock);
return ret;
}
+static void guc_client_update_wq_rsvd(struct i915_guc_client *client, int size)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&client->wq_lock, flags);
+ client->wq_rsvd += size;
+ spin_unlock_irqrestore(&client->wq_lock, flags);
+}
+
void i915_guc_wq_unreserve(struct drm_i915_gem_request *request)
{
- const size_t wqi_size = sizeof(struct guc_wq_item);
+ const int wqi_size = sizeof(struct guc_wq_item);
struct i915_guc_client *client = request->i915->guc.execbuf_client;
GEM_BUG_ON(READ_ONCE(client->wq_rsvd) < wqi_size);
-
- spin_lock(&client->wq_lock);
- client->wq_rsvd -= wqi_size;
- spin_unlock(&client->wq_lock);
+ guc_client_update_wq_rsvd(client, -wqi_size);
}
/* Construct a Work Item and append it to the GuC's Work Queue */
unsigned int engine_id = engine->id;
struct intel_guc *guc = &rq->i915->guc;
struct i915_guc_client *client = guc->execbuf_client;
+ unsigned long flags;
int b_ret;
- spin_lock(&client->wq_lock);
- guc_wq_item_append(client, rq);
-
/* WA to flush out the pending GMADR writes to ring buffer. */
if (i915_vma_is_map_and_fenceable(rq->ring->vma))
POSTING_READ_FW(GUC_STATUS);
+ spin_lock_irqsave(&client->wq_lock, flags);
+
+ guc_wq_item_append(client, rq);
b_ret = guc_ring_doorbell(client);
client->submissions[engine_id] += 1;
guc->submissions[engine_id] += 1;
guc->last_seqno[engine_id] = rq->global_seqno;
- spin_unlock(&client->wq_lock);
+
+ spin_unlock_irqrestore(&client->wq_lock, flags);
}
static void i915_guc_submit(struct drm_i915_gem_request *rq)
{
- i915_gem_request_submit(rq);
+ __i915_gem_request_submit(rq);
__i915_guc_submit(rq);
}
+static void nested_enable_signaling(struct drm_i915_gem_request *rq)
+{
+ /* If we use dma_fence_enable_sw_signaling() directly, lockdep
+ * detects an ordering issue between the fence lockclass and the
+ * global_timeline. This circular dependency can only occur via 2
+ * different fences (but same fence lockclass), so we use the nesting
+ * annotation here to prevent the warn, equivalent to the nesting
+ * inside i915_gem_request_submit() for when we also enable the
+ * signaler.
+ */
+
+ if (test_and_set_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT,
+ &rq->fence.flags))
+ return;
+
+ GEM_BUG_ON(test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &rq->fence.flags));
+ trace_dma_fence_enable_signal(&rq->fence);
+
+ spin_lock_nested(&rq->lock, SINGLE_DEPTH_NESTING);
+ intel_engine_enable_signaling(rq);
+ spin_unlock(&rq->lock);
+}
+
+static bool i915_guc_dequeue(struct intel_engine_cs *engine)
+{
+ struct execlist_port *port = engine->execlist_port;
+ struct drm_i915_gem_request *last = port[0].request;
+ unsigned long flags;
+ struct rb_node *rb;
+ bool submit = false;
+
+ /* After execlist_first is updated, the tasklet will be rescheduled.
+ *
+ * If we are currently running (inside the tasklet) and a third
+ * party queues a request and so updates engine->execlist_first under
+ * the spinlock (which we have elided), it will atomically set the
+ * TASKLET_SCHED flag causing the us to be re-executed and pick up
+ * the change in state (the update to TASKLET_SCHED incurs a memory
+ * barrier making this cross-cpu checking safe).
+ */
+ if (!READ_ONCE(engine->execlist_first))
+ return false;
+
+ spin_lock_irqsave(&engine->timeline->lock, flags);
+ rb = engine->execlist_first;
+ while (rb) {
+ struct drm_i915_gem_request *rq =
+ rb_entry(rb, typeof(*rq), priotree.node);
+
+ if (last && rq->ctx != last->ctx) {
+ if (port != engine->execlist_port)
+ break;
+
+ i915_gem_request_assign(&port->request, last);
+ nested_enable_signaling(last);
+ port++;
+ }
+
+ rb = rb_next(rb);
+ rb_erase(&rq->priotree.node, &engine->execlist_queue);
+ RB_CLEAR_NODE(&rq->priotree.node);
+ rq->priotree.priority = INT_MAX;
+
+ trace_i915_gem_request_in(rq, port - engine->execlist_port);
+ i915_guc_submit(rq);
+ last = rq;
+ submit = true;
+ }
+ if (submit) {
+ i915_gem_request_assign(&port->request, last);
+ nested_enable_signaling(last);
+ engine->execlist_first = rb;
+ }
+ spin_unlock_irqrestore(&engine->timeline->lock, flags);
+
+ return submit;
+}
+
+static void i915_guc_irq_handler(unsigned long data)
+{
+ struct intel_engine_cs *engine = (struct intel_engine_cs *)data;
+ struct execlist_port *port = engine->execlist_port;
+ struct drm_i915_gem_request *rq;
+ bool submit;
+
+ do {
+ rq = port[0].request;
+ while (rq && i915_gem_request_completed(rq)) {
+ trace_i915_gem_request_out(rq);
+ i915_gem_request_put(rq);
+ port[0].request = port[1].request;
+ port[1].request = NULL;
+ rq = port[0].request;
+ }
+
+ submit = false;
+ if (!port[1].request)
+ submit = i915_guc_dequeue(engine);
+ } while (submit);
+}
+
/*
* Everything below here is concerned with setup & teardown, and is
* therefore not part of the somewhat time-critical batch-submission
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
struct i915_vma *vma;
- struct guc_ads *ads;
- struct guc_policies *policies;
- struct guc_mmio_reg_state *reg_state;
- struct intel_engine_cs *engine;
- enum intel_engine_id id;
struct page *page;
- u32 size;
-
/* The ads obj includes the struct itself and buffers passed to GuC */
- size = sizeof(struct guc_ads) + sizeof(struct guc_policies) +
- sizeof(struct guc_mmio_reg_state) +
- GUC_S3_SAVE_SPACE_PAGES * PAGE_SIZE;
+ struct {
+ struct guc_ads ads;
+ struct guc_policies policies;
+ struct guc_mmio_reg_state reg_state;
+ u8 reg_state_buffer[GUC_S3_SAVE_SPACE_PAGES * PAGE_SIZE];
+ } __packed *blob;
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ u32 base;
vma = guc->ads_vma;
if (!vma) {
- vma = intel_guc_allocate_vma(guc, PAGE_ALIGN(size));
+ vma = intel_guc_allocate_vma(guc, PAGE_ALIGN(sizeof(*blob)));
if (IS_ERR(vma))
return;
}
page = i915_vma_first_page(vma);
- ads = kmap(page);
-
- /*
- * The GuC requires a "Golden Context" when it reinitialises
- * engines after a reset. Here we use the Render ring default
- * context, which must already exist and be pinned in the GGTT,
- * so its address won't change after we've told the GuC where
- * to find it.
- */
- engine = dev_priv->engine[RCS];
- ads->golden_context_lrca = engine->status_page.ggtt_offset;
-
- for_each_engine(engine, dev_priv, id)
- ads->eng_state_size[engine->guc_id] = intel_lr_context_size(engine);
+ blob = kmap(page);
/* GuC scheduling policies */
- policies = (void *)ads + sizeof(struct guc_ads);
- guc_policies_init(policies);
-
- ads->scheduler_policies =
- guc_ggtt_offset(vma) + sizeof(struct guc_ads);
+ guc_policies_init(&blob->policies);
/* MMIO reg state */
- reg_state = (void *)policies + sizeof(struct guc_policies);
-
for_each_engine(engine, dev_priv, id) {
- reg_state->mmio_white_list[engine->guc_id].mmio_start =
+ blob->reg_state.mmio_white_list[engine->guc_id].mmio_start =
engine->mmio_base + GUC_MMIO_WHITE_LIST_START;
/* Nothing to be saved or restored for now. */
- reg_state->mmio_white_list[engine->guc_id].count = 0;
+ blob->reg_state.mmio_white_list[engine->guc_id].count = 0;
}
- ads->reg_state_addr = ads->scheduler_policies +
- sizeof(struct guc_policies);
+ /*
+ * The GuC requires a "Golden Context" when it reinitialises
+ * engines after a reset. Here we use the Render ring default
+ * context, which must already exist and be pinned in the GGTT,
+ * so its address won't change after we've told the GuC where
+ * to find it.
+ */
+ blob->ads.golden_context_lrca =
+ dev_priv->engine[RCS]->status_page.ggtt_offset;
+
+ for_each_engine(engine, dev_priv, id)
+ blob->ads.eng_state_size[engine->guc_id] =
+ intel_lr_context_size(engine);
- ads->reg_state_buffer = ads->reg_state_addr +
- sizeof(struct guc_mmio_reg_state);
+ base = guc_ggtt_offset(vma);
+ blob->ads.scheduler_policies = base + ptr_offset(blob, policies);
+ blob->ads.reg_state_buffer = base + ptr_offset(blob, reg_state_buffer);
+ blob->ads.reg_state_addr = base + ptr_offset(blob, reg_state);
kunmap(page);
}
client->wq_tail = 0;
}
+static void guc_interrupts_capture(struct drm_i915_private *dev_priv)
+{
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ int irqs;
+
+ /* tell all command streamers to forward interrupts (but not vblank) to GuC */
+ irqs = _MASKED_BIT_ENABLE(GFX_INTERRUPT_STEERING);
+ for_each_engine(engine, dev_priv, id)
+ I915_WRITE(RING_MODE_GEN7(engine), irqs);
+
+ /* route USER_INTERRUPT to Host, all others are sent to GuC. */
+ irqs = GT_RENDER_USER_INTERRUPT << GEN8_RCS_IRQ_SHIFT |
+ GT_RENDER_USER_INTERRUPT << GEN8_BCS_IRQ_SHIFT;
+ /* These three registers have the same bit definitions */
+ I915_WRITE(GUC_BCS_RCS_IER, ~irqs);
+ I915_WRITE(GUC_VCS2_VCS1_IER, ~irqs);
+ I915_WRITE(GUC_WD_VECS_IER, ~irqs);
+
+ /*
+ * The REDIRECT_TO_GUC bit of the PMINTRMSK register directs all
+ * (unmasked) PM interrupts to the GuC. All other bits of this
+ * register *disable* generation of a specific interrupt.
+ *
+ * 'pm_intrmsk_mbz' indicates bits that are NOT to be set when
+ * writing to the PM interrupt mask register, i.e. interrupts
+ * that must not be disabled.
+ *
+ * If the GuC is handling these interrupts, then we must not let
+ * the PM code disable ANY interrupt that the GuC is expecting.
+ * So for each ENABLED (0) bit in this register, we must SET the
+ * bit in pm_intrmsk_mbz so that it's left enabled for the GuC.
+ * GuC needs ARAT expired interrupt unmasked hence it is set in
+ * pm_intrmsk_mbz.
+ *
+ * Here we CLEAR REDIRECT_TO_GUC bit in pm_intrmsk_mbz, which will
+ * result in the register bit being left SET!
+ */
+ dev_priv->rps.pm_intrmsk_mbz |= ARAT_EXPIRED_INTRMSK;
+ dev_priv->rps.pm_intrmsk_mbz &= ~GEN8_PMINTR_DISABLE_REDIRECT_TO_GUC;
+}
+
int i915_guc_submission_enable(struct drm_i915_private *dev_priv)
{
struct intel_guc *guc = &dev_priv->guc;
guc_init_doorbell_hw(guc);
/* Take over from manual control of ELSP (execlists) */
+ guc_interrupts_capture(dev_priv);
+
for_each_engine(engine, dev_priv, id) {
+ const int wqi_size = sizeof(struct guc_wq_item);
struct drm_i915_gem_request *rq;
- engine->submit_request = i915_guc_submit;
- engine->schedule = NULL;
+ /* The tasklet was initialised by execlists, and may be in
+ * a state of flux (across a reset) and so we just want to
+ * take over the callback without changing any other state
+ * in the tasklet.
+ */
+ engine->irq_tasklet.func = i915_guc_irq_handler;
+ clear_bit(ENGINE_IRQ_EXECLIST, &engine->irq_posted);
/* Replay the current set of previously submitted requests */
+ spin_lock_irq(&engine->timeline->lock);
list_for_each_entry(rq, &engine->timeline->requests, link) {
- client->wq_rsvd += sizeof(struct guc_wq_item);
+ guc_client_update_wq_rsvd(client, wqi_size);
__i915_guc_submit(rq);
}
+ spin_unlock_irq(&engine->timeline->lock);
}
return 0;
}
+static void guc_interrupts_release(struct drm_i915_private *dev_priv)
+{
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ int irqs;
+
+ /*
+ * tell all command streamers NOT to forward interrupts or vblank
+ * to GuC.
+ */
+ irqs = _MASKED_FIELD(GFX_FORWARD_VBLANK_MASK, GFX_FORWARD_VBLANK_NEVER);
+ irqs |= _MASKED_BIT_DISABLE(GFX_INTERRUPT_STEERING);
+ for_each_engine(engine, dev_priv, id)
+ I915_WRITE(RING_MODE_GEN7(engine), irqs);
+
+ /* route all GT interrupts to the host */
+ I915_WRITE(GUC_BCS_RCS_IER, 0);
+ I915_WRITE(GUC_VCS2_VCS1_IER, 0);
+ I915_WRITE(GUC_WD_VECS_IER, 0);
+
+ dev_priv->rps.pm_intrmsk_mbz |= GEN8_PMINTR_DISABLE_REDIRECT_TO_GUC;
+ dev_priv->rps.pm_intrmsk_mbz &= ~ARAT_EXPIRED_INTRMSK;
+}
+
void i915_guc_submission_disable(struct drm_i915_private *dev_priv)
{
struct intel_guc *guc = &dev_priv->guc;
+ guc_interrupts_release(dev_priv);
+
if (!guc->execbuf_client)
return;
/* Revert back to manual ELSP submission */
- intel_execlists_enable_submission(dev_priv);
+ intel_engines_reset_default_submission(dev_priv);
}
void i915_guc_submission_fini(struct drm_i915_private *dev_priv)
{
uint32_t val;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
WARN_ON(bits & ~mask);
val = I915_READ(PORT_HOTPLUG_EN);
{
uint32_t new_val;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
WARN_ON(enabled_irq_mask & ~interrupt_mask);
uint32_t interrupt_mask,
uint32_t enabled_irq_mask)
{
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
WARN_ON(enabled_irq_mask & ~interrupt_mask);
WARN_ON(enabled_irq_mask & ~interrupt_mask);
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
new_val = dev_priv->pm_imr;
new_val &= ~interrupt_mask;
{
i915_reg_t reg = gen6_pm_iir(dev_priv);
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
I915_WRITE(reg, reset_mask);
I915_WRITE(reg, reset_mask);
void gen6_enable_pm_irq(struct drm_i915_private *dev_priv, u32 enable_mask)
{
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
dev_priv->pm_ier |= enable_mask;
I915_WRITE(gen6_pm_ier(dev_priv), dev_priv->pm_ier);
void gen6_disable_pm_irq(struct drm_i915_private *dev_priv, u32 disable_mask)
{
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
dev_priv->pm_ier &= ~disable_mask;
__gen6_mask_pm_irq(dev_priv, disable_mask);
spin_unlock_irq(&dev_priv->irq_lock);
}
-u32 gen6_sanitize_rps_pm_mask(struct drm_i915_private *dev_priv, u32 mask)
-{
- return (mask & ~dev_priv->rps.pm_intr_keep);
-}
-
void gen6_disable_rps_interrupts(struct drm_i915_private *dev_priv)
{
if (!READ_ONCE(dev_priv->rps.interrupts_enabled))
uint32_t new_val;
uint32_t old_val;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
WARN_ON(enabled_irq_mask & ~interrupt_mask);
{
uint32_t new_val;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
WARN_ON(enabled_irq_mask & ~interrupt_mask);
WARN_ON(enabled_irq_mask & ~interrupt_mask);
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
if (WARN_ON(!intel_irqs_enabled(dev_priv)))
return;
i915_reg_t reg = PIPESTAT(pipe);
u32 pipestat = I915_READ(reg) & PIPESTAT_INT_ENABLE_MASK;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
WARN_ON(!intel_irqs_enabled(dev_priv));
if (WARN_ONCE(enable_mask & ~PIPESTAT_INT_ENABLE_MASK ||
i915_reg_t reg = PIPESTAT(pipe);
u32 pipestat = I915_READ(reg) & PIPESTAT_INT_ENABLE_MASK;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
WARN_ON(!intel_irqs_enabled(dev_priv));
if (WARN_ONCE(enable_mask & ~PIPESTAT_INT_ENABLE_MASK ||
struct intel_crtc *intel_crtc = intel_get_crtc_for_pipe(dev_priv,
pipe);
const struct drm_display_mode *mode = &intel_crtc->base.hwmode;
+ unsigned long irqflags;
htotal = mode->crtc_htotal;
hsync_start = mode->crtc_hsync_start;
high_frame = PIPEFRAME(pipe);
low_frame = PIPEFRAMEPIXEL(pipe);
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
/*
* High & low register fields aren't synchronized, so make sure
* we get a low value that's stable across two reads of the high
* register.
*/
do {
- high1 = I915_READ(high_frame) & PIPE_FRAME_HIGH_MASK;
- low = I915_READ(low_frame);
- high2 = I915_READ(high_frame) & PIPE_FRAME_HIGH_MASK;
+ high1 = I915_READ_FW(high_frame) & PIPE_FRAME_HIGH_MASK;
+ low = I915_READ_FW(low_frame);
+ high2 = I915_READ_FW(high_frame) & PIPE_FRAME_HIGH_MASK;
} while (high1 != high2);
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
+
high1 >>= PIPE_FRAME_HIGH_SHIFT;
pixel = low & PIPE_PIXEL_MASK;
low >>= PIPE_FRAME_LOW_SHIFT;
enum pipe pipe = crtc->pipe;
int position, vtotal;
+ if (!crtc->active)
+ return -1;
+
vtotal = mode->crtc_vtotal;
if (mode->flags & DRM_MODE_FLAG_INTERLACE)
vtotal /= 2;
for (i = 0; i < 100; i++) {
udelay(1);
- temp = __raw_i915_read32(dev_priv, PIPEDSL(pipe)) &
- DSL_LINEMASK_GEN3;
+ temp = I915_READ_FW(PIPEDSL(pipe)) & DSL_LINEMASK_GEN3;
if (temp != position) {
position = temp;
break;
static void notify_ring(struct intel_engine_cs *engine)
{
- smp_store_mb(engine->breadcrumbs.irq_posted, true);
- if (intel_engine_wakeup(engine))
- trace_i915_gem_request_notify(engine);
+ struct drm_i915_gem_request *rq = NULL;
+ struct intel_wait *wait;
+
+ atomic_inc(&engine->irq_count);
+ set_bit(ENGINE_IRQ_BREADCRUMB, &engine->irq_posted);
+
+ spin_lock(&engine->breadcrumbs.irq_lock);
+ wait = engine->breadcrumbs.irq_wait;
+ if (wait) {
+ /* We use a callback from the dma-fence to submit
+ * requests after waiting on our own requests. To
+ * ensure minimum delay in queuing the next request to
+ * hardware, signal the fence now rather than wait for
+ * the signaler to be woken up. We still wake up the
+ * waiter in order to handle the irq-seqno coherency
+ * issues (we may receive the interrupt before the
+ * seqno is written, see __i915_request_irq_complete())
+ * and to handle coalescing of multiple seqno updates
+ * and many waiters.
+ */
+ if (i915_seqno_passed(intel_engine_get_seqno(engine),
+ wait->seqno) &&
+ !test_bit(DMA_FENCE_FLAG_SIGNALED_BIT,
+ &wait->request->fence.flags))
+ rq = i915_gem_request_get(wait->request);
+
+ wake_up_process(wait->tsk);
+ } else {
+ __intel_engine_disarm_breadcrumbs(engine);
+ }
+ spin_unlock(&engine->breadcrumbs.irq_lock);
+
+ if (rq) {
+ dma_fence_signal(&rq->fence);
+ i915_gem_request_put(rq);
+ }
+
+ trace_intel_engine_notify(engine, wait);
}
static void vlv_c0_read(struct drm_i915_private *dev_priv,
struct intel_rps_ei *ei)
{
- ei->cz_clock = vlv_punit_read(dev_priv, PUNIT_REG_CZ_TIMESTAMP);
+ ei->ktime = ktime_get_raw();
ei->render_c0 = I915_READ(VLV_RENDER_C0_COUNT);
ei->media_c0 = I915_READ(VLV_MEDIA_C0_COUNT);
}
return 0;
vlv_c0_read(dev_priv, &now);
- if (now.cz_clock == 0)
- return 0;
- if (prev->cz_clock) {
+ if (prev->ktime) {
u64 time, c0;
- unsigned int mul;
+ u32 render, media;
- mul = VLV_CZ_CLOCK_TO_MILLI_SEC * 100; /* scale to threshold% */
- if (I915_READ(VLV_COUNTER_CONTROL) & VLV_COUNT_RANGE_HIGH)
- mul <<= 8;
+ time = ktime_us_delta(now.ktime, prev->ktime);
- time = now.cz_clock - prev->cz_clock;
time *= dev_priv->czclk_freq;
/* Workload can be split between render + media,
* mesa. To account for this we need to combine both engines
* into our activity counter.
*/
- c0 = now.render_c0 - prev->render_c0;
- c0 += now.media_c0 - prev->media_c0;
- c0 *= mul;
+ render = now.render_c0 - prev->render_c0;
+ media = now.media_c0 - prev->media_c0;
+ c0 = max(render, media);
+ c0 *= 1000 * 100 << 8; /* to usecs and scale to threshold% */
if (c0 > time * dev_priv->rps.up_threshold)
events = GEN6_PM_RP_UP_THRESHOLD;
{
struct drm_i915_private *dev_priv =
container_of(work, struct drm_i915_private, rps.work);
- bool client_boost;
+ bool client_boost = false;
int new_delay, adj, min, max;
- u32 pm_iir;
+ u32 pm_iir = 0;
spin_lock_irq(&dev_priv->irq_lock);
- /* Speed up work cancelation during disabling rps interrupts. */
- if (!dev_priv->rps.interrupts_enabled) {
- spin_unlock_irq(&dev_priv->irq_lock);
- return;
+ if (dev_priv->rps.interrupts_enabled) {
+ pm_iir = fetch_and_zero(&dev_priv->rps.pm_iir);
+ client_boost = fetch_and_zero(&dev_priv->rps.client_boost);
}
-
- pm_iir = dev_priv->rps.pm_iir;
- dev_priv->rps.pm_iir = 0;
- /* Make sure not to corrupt PMIMR state used by ringbuffer on GEN6 */
- gen6_unmask_pm_irq(dev_priv, dev_priv->pm_rps_events);
- client_boost = dev_priv->rps.client_boost;
- dev_priv->rps.client_boost = false;
spin_unlock_irq(&dev_priv->irq_lock);
/* Make sure we didn't queue anything we're not going to process. */
WARN_ON(pm_iir & ~dev_priv->pm_rps_events);
-
if ((pm_iir & dev_priv->pm_rps_events) == 0 && !client_boost)
- return;
+ goto out;
mutex_lock(&dev_priv->rps.hw_lock);
if (new_delay >= dev_priv->rps.max_freq_softlimit)
adj = 0;
- /*
- * For better performance, jump directly
- * to RPe if we're below it.
- */
- if (new_delay < dev_priv->rps.efficient_freq - adj) {
- new_delay = dev_priv->rps.efficient_freq;
- adj = 0;
- }
} else if (client_boost || any_waiters(dev_priv)) {
adj = 0;
} else if (pm_iir & GEN6_PM_RP_DOWN_TIMEOUT) {
if (dev_priv->rps.cur_freq > dev_priv->rps.efficient_freq)
new_delay = dev_priv->rps.efficient_freq;
- else
+ else if (dev_priv->rps.cur_freq > dev_priv->rps.min_freq_softlimit)
new_delay = dev_priv->rps.min_freq_softlimit;
adj = 0;
} else if (pm_iir & GEN6_PM_RP_DOWN_THRESHOLD) {
new_delay += adj;
new_delay = clamp_t(int, new_delay, min, max);
- intel_set_rps(dev_priv, new_delay);
+ if (intel_set_rps(dev_priv, new_delay)) {
+ DRM_DEBUG_DRIVER("Failed to set new GPU frequency\n");
+ dev_priv->rps.last_adj = 0;
+ }
mutex_unlock(&dev_priv->rps.hw_lock);
+
+out:
+ /* Make sure not to corrupt PMIMR state used by ringbuffer on GEN6 */
+ spin_lock_irq(&dev_priv->irq_lock);
+ if (dev_priv->rps.interrupts_enabled)
+ gen6_unmask_pm_irq(dev_priv, dev_priv->pm_rps_events);
+ spin_unlock_irq(&dev_priv->irq_lock);
}
static __always_inline void
gen8_cs_irq_handler(struct intel_engine_cs *engine, u32 iir, int test_shift)
{
- if (iir & (GT_RENDER_USER_INTERRUPT << test_shift))
+ bool tasklet = false;
+
+ if (iir & (GT_CONTEXT_SWITCH_INTERRUPT << test_shift)) {
+ set_bit(ENGINE_IRQ_EXECLIST, &engine->irq_posted);
+ tasklet = true;
+ }
+
+ if (iir & (GT_RENDER_USER_INTERRUPT << test_shift)) {
notify_ring(engine);
- if (iir & (GT_CONTEXT_SWITCH_INTERRUPT << test_shift))
- tasklet_schedule(&engine->irq_tasklet);
+ tasklet |= i915.enable_guc_submission;
+ }
+
+ if (tasklet)
+ tasklet_hi_schedule(&engine->irq_tasklet);
}
static irqreturn_t gen8_gt_irq_ack(struct drm_i915_private *dev_priv,
return ret;
}
-static void i915_error_wake_up(struct drm_i915_private *dev_priv)
-{
- /*
- * Notify all waiters for GPU completion events that reset state has
- * been changed, and that they need to restart their wait after
- * checking for potential errors (and bail out to drop locks if there is
- * a gpu reset pending so that i915_error_work_func can acquire them).
- */
-
- /* Wake up __wait_seqno, potentially holding dev->struct_mutex. */
- wake_up_all(&dev_priv->gpu_error.wait_queue);
-
- /* Wake up intel_crtc_wait_for_pending_flips, holding crtc->mutex. */
- wake_up_all(&dev_priv->pending_flip_queue);
-}
-
/**
* i915_reset_and_wakeup - do process context error handling work
* @dev_priv: i915 device private
DRM_DEBUG_DRIVER("resetting chip\n");
kobject_uevent_env(kobj, KOBJ_CHANGE, reset_event);
- /*
- * In most cases it's guaranteed that we get here with an RPM
- * reference held, for example because there is a pending GPU
- * request that won't finish until the reset is done. This
- * isn't the case at least when we get here by doing a
- * simulated reset via debugs, so get an RPM reference.
- */
- intel_runtime_pm_get(dev_priv);
intel_prepare_reset(dev_priv);
+ set_bit(I915_RESET_HANDOFF, &dev_priv->gpu_error.flags);
+ wake_up_all(&dev_priv->gpu_error.wait_queue);
+
do {
/*
* All state reset _must_ be completed before we update the
/* We need to wait for anyone holding the lock to wakeup */
} while (wait_on_bit_timeout(&dev_priv->gpu_error.flags,
- I915_RESET_IN_PROGRESS,
+ I915_RESET_HANDOFF,
TASK_UNINTERRUPTIBLE,
HZ));
intel_finish_reset(dev_priv);
- intel_runtime_pm_put(dev_priv);
if (!test_bit(I915_WEDGED, &dev_priv->gpu_error.flags))
kobject_uevent_env(kobj,
* Note: The wake_up also serves as a memory barrier so that
* waiters see the updated value of the dev_priv->gpu_error.
*/
+ clear_bit(I915_RESET_BACKOFF, &dev_priv->gpu_error.flags);
wake_up_all(&dev_priv->gpu_error.reset_queue);
}
vscnprintf(error_msg, sizeof(error_msg), fmt, args);
va_end(args);
+ /*
+ * In most cases it's guaranteed that we get here with an RPM
+ * reference held, for example because there is a pending GPU
+ * request that won't finish until the reset is done. This
+ * isn't the case at least when we get here by doing a
+ * simulated reset via debugfs, so get an RPM reference.
+ */
+ intel_runtime_pm_get(dev_priv);
+
i915_capture_error_state(dev_priv, engine_mask, error_msg);
i915_clear_error_registers(dev_priv);
if (!engine_mask)
- return;
+ goto out;
- if (test_and_set_bit(I915_RESET_IN_PROGRESS,
+ if (test_and_set_bit(I915_RESET_BACKOFF,
&dev_priv->gpu_error.flags))
- return;
-
- /*
- * Wakeup waiting processes so that the reset function
- * i915_reset_and_wakeup doesn't deadlock trying to grab
- * various locks. By bumping the reset counter first, the woken
- * processes will see a reset in progress and back off,
- * releasing their locks and then wait for the reset completion.
- * We must do this for _all_ gpu waiters that might hold locks
- * that the reset work needs to acquire.
- *
- * Note: The wake_up also provides a memory barrier to ensure that the
- * waiters see the updated value of the reset flags.
- */
- i915_error_wake_up(dev_priv);
+ goto out;
i915_reset_and_wakeup(dev_priv);
+
+out:
+ intel_runtime_pm_put(dev_priv);
}
/* Called from drm generic code, passed 'crtc' which
return enabled_irqs;
}
-static void ibx_hpd_irq_setup(struct drm_i915_private *dev_priv)
+static void ibx_hpd_detection_setup(struct drm_i915_private *dev_priv)
{
- u32 hotplug_irqs, hotplug, enabled_irqs;
-
- if (HAS_PCH_IBX(dev_priv)) {
- hotplug_irqs = SDE_HOTPLUG_MASK;
- enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_ibx);
- } else {
- hotplug_irqs = SDE_HOTPLUG_MASK_CPT;
- enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_cpt);
- }
-
- ibx_display_interrupt_update(dev_priv, hotplug_irqs, enabled_irqs);
+ u32 hotplug;
/*
* Enable digital hotplug on the PCH, and configure the DP short pulse
* The pulse duration bits are reserved on LPT+.
*/
hotplug = I915_READ(PCH_PORT_HOTPLUG);
- hotplug &= ~(PORTD_PULSE_DURATION_MASK|PORTC_PULSE_DURATION_MASK|PORTB_PULSE_DURATION_MASK);
- hotplug |= PORTD_HOTPLUG_ENABLE | PORTD_PULSE_DURATION_2ms;
- hotplug |= PORTC_HOTPLUG_ENABLE | PORTC_PULSE_DURATION_2ms;
+ hotplug &= ~(PORTB_PULSE_DURATION_MASK |
+ PORTC_PULSE_DURATION_MASK |
+ PORTD_PULSE_DURATION_MASK);
hotplug |= PORTB_HOTPLUG_ENABLE | PORTB_PULSE_DURATION_2ms;
+ hotplug |= PORTC_HOTPLUG_ENABLE | PORTC_PULSE_DURATION_2ms;
+ hotplug |= PORTD_HOTPLUG_ENABLE | PORTD_PULSE_DURATION_2ms;
/*
* When CPU and PCH are on the same package, port A
* HPD must be enabled in both north and south.
I915_WRITE(PCH_PORT_HOTPLUG, hotplug);
}
+static void ibx_hpd_irq_setup(struct drm_i915_private *dev_priv)
+{
+ u32 hotplug_irqs, enabled_irqs;
+
+ if (HAS_PCH_IBX(dev_priv)) {
+ hotplug_irqs = SDE_HOTPLUG_MASK;
+ enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_ibx);
+ } else {
+ hotplug_irqs = SDE_HOTPLUG_MASK_CPT;
+ enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_cpt);
+ }
+
+ ibx_display_interrupt_update(dev_priv, hotplug_irqs, enabled_irqs);
+
+ ibx_hpd_detection_setup(dev_priv);
+}
+
static void spt_hpd_detection_setup(struct drm_i915_private *dev_priv)
{
u32 hotplug;
spt_hpd_detection_setup(dev_priv);
}
+static void ilk_hpd_detection_setup(struct drm_i915_private *dev_priv)
+{
+ u32 hotplug;
+
+ /*
+ * Enable digital hotplug on the CPU, and configure the DP short pulse
+ * duration to 2ms (which is the minimum in the Display Port spec)
+ * The pulse duration bits are reserved on HSW+.
+ */
+ hotplug = I915_READ(DIGITAL_PORT_HOTPLUG_CNTRL);
+ hotplug &= ~DIGITAL_PORTA_PULSE_DURATION_MASK;
+ hotplug |= DIGITAL_PORTA_HOTPLUG_ENABLE |
+ DIGITAL_PORTA_PULSE_DURATION_2ms;
+ I915_WRITE(DIGITAL_PORT_HOTPLUG_CNTRL, hotplug);
+}
+
static void ilk_hpd_irq_setup(struct drm_i915_private *dev_priv)
{
- u32 hotplug_irqs, hotplug, enabled_irqs;
+ u32 hotplug_irqs, enabled_irqs;
if (INTEL_GEN(dev_priv) >= 8) {
hotplug_irqs = GEN8_PORT_DP_A_HOTPLUG;
ilk_update_display_irq(dev_priv, hotplug_irqs, enabled_irqs);
}
- /*
- * Enable digital hotplug on the CPU, and configure the DP short pulse
- * duration to 2ms (which is the minimum in the Display Port spec)
- * The pulse duration bits are reserved on HSW+.
- */
- hotplug = I915_READ(DIGITAL_PORT_HOTPLUG_CNTRL);
- hotplug &= ~DIGITAL_PORTA_PULSE_DURATION_MASK;
- hotplug |= DIGITAL_PORTA_HOTPLUG_ENABLE | DIGITAL_PORTA_PULSE_DURATION_2ms;
- I915_WRITE(DIGITAL_PORT_HOTPLUG_CNTRL, hotplug);
+ ilk_hpd_detection_setup(dev_priv);
ibx_hpd_irq_setup(dev_priv);
}
if (HAS_PCH_IBX(dev_priv) || HAS_PCH_CPT(dev_priv) ||
HAS_PCH_LPT(dev_priv))
- ; /* TODO: Enable HPD detection on older PCH platforms too */
+ ibx_hpd_detection_setup(dev_priv);
else
spt_hpd_detection_setup(dev_priv);
}
gen5_gt_irq_postinstall(dev);
+ ilk_hpd_detection_setup(dev_priv);
+
ibx_irq_postinstall(dev);
if (IS_IRONLAKE_M(dev_priv)) {
void valleyview_enable_display_irqs(struct drm_i915_private *dev_priv)
{
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
if (dev_priv->display_irqs_enabled)
return;
void valleyview_disable_display_irqs(struct drm_i915_private *dev_priv)
{
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
if (!dev_priv->display_irqs_enabled)
return;
if (IS_GEN9_LP(dev_priv))
bxt_hpd_detection_setup(dev_priv);
+ else if (IS_BROADWELL(dev_priv))
+ ilk_hpd_detection_setup(dev_priv);
}
static int gen8_irq_postinstall(struct drm_device *dev)
{
u32 hotplug_en;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
/* Note HDMI and DP share hotplug bits */
/* enable bits are the same for all generations */
else
dev_priv->pm_rps_events = GEN6_PM_RPS_EVENTS;
- dev_priv->rps.pm_intr_keep = 0;
+ dev_priv->rps.pm_intrmsk_mbz = 0;
/*
* SNB,IVB can while VLV,CHV may hard hang on looping batchbuffer
* TODO: verify if this can be reproduced on VLV,CHV.
*/
if (INTEL_INFO(dev_priv)->gen <= 7 && !IS_HASWELL(dev_priv))
- dev_priv->rps.pm_intr_keep |= GEN6_PM_RP_UP_EI_EXPIRED;
+ dev_priv->rps.pm_intrmsk_mbz |= GEN6_PM_RP_UP_EI_EXPIRED;
if (INTEL_INFO(dev_priv)->gen >= 8)
- dev_priv->rps.pm_intr_keep |= GEN8_PMINTR_REDIRECT_TO_GUC;
+ dev_priv->rps.pm_intrmsk_mbz |= GEN8_PMINTR_DISABLE_REDIRECT_TO_GUC;
if (IS_GEN2(dev_priv)) {
/* Gen2 doesn't have a hardware frame counter */
dev->max_vblank_count = 0;
- dev->driver->get_vblank_counter = drm_vblank_no_hw_counter;
} else if (IS_G4X(dev_priv) || INTEL_INFO(dev_priv)->gen >= 5) {
dev->max_vblank_count = 0xffffffff; /* full 32 bit counter */
dev->driver->get_vblank_counter = g4x_get_vblank_counter;
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
dev_priv->display_irqs_enabled = false;
+ dev_priv->hotplug.hpd_storm_threshold = HPD_STORM_DEFAULT_THRESHOLD;
+
dev->driver->get_vblank_timestamp = i915_get_vblank_timestamp;
dev->driver->get_scanout_position = i915_get_crtc_scanoutpos;
.enable_guc_loading = 0,
.enable_guc_submission = 0,
.guc_log_level = -1,
+ .guc_firmware_path = NULL,
+ .huc_firmware_path = NULL,
.enable_dp_mst = true,
.inject_load_failure = 0,
.enable_dpcd_backlight = false,
"(0=disabled, 1=enabled - link mode chosen per-platform, 2=force link-standby mode, 3=force link-off mode) "
"Default: -1 (use per-chip default)");
-module_param_named_unsafe(alpha_support, i915.alpha_support, int, 0400);
+module_param_named_unsafe(alpha_support, i915.alpha_support, bool, 0400);
MODULE_PARM_DESC(alpha_support,
"Enable alpha quality driver support for latest hardware. "
"See also CONFIG_DRM_I915_ALPHA_SUPPORT.");
MODULE_PARM_DESC(verbose_state_checks,
"Enable verbose logs (ie. WARN_ON()) in case of unexpected hw state conditions.");
-module_param_named_unsafe(nuclear_pageflip, i915.nuclear_pageflip, bool, 0600);
+module_param_named_unsafe(nuclear_pageflip, i915.nuclear_pageflip, bool, 0400);
MODULE_PARM_DESC(nuclear_pageflip,
- "Force atomic modeset functionality; asynchronous mode is not yet supported. (default: false).");
+ "Force enable atomic functionality on platforms that don't have full support yet.");
/* WA to get away with the default setting in VBT for early platforms.Will be removed */
module_param_named_unsafe(edp_vswing, i915.edp_vswing, int, 0400);
MODULE_PARM_DESC(guc_log_level,
"GuC firmware logging level (-1:disabled (default), 0-3:enabled)");
+module_param_named_unsafe(guc_firmware_path, i915.guc_firmware_path, charp, 0400);
+MODULE_PARM_DESC(guc_firmware_path,
+ "GuC firmware path to use instead of the default one");
+
+module_param_named_unsafe(huc_firmware_path, i915.huc_firmware_path, charp, 0400);
+MODULE_PARM_DESC(huc_firmware_path,
+ "HuC firmware path to use instead of the default one");
+
module_param_named_unsafe(enable_dp_mst, i915.enable_dp_mst, bool, 0600);
MODULE_PARM_DESC(enable_dp_mst,
"Enable multi-stream transport (MST) for new DisplayPort sinks. (default: true)");
#include <linux/cache.h> /* for __read_mostly */
+#define I915_PARAMS_FOR_EACH(func) \
+ func(int, modeset); \
+ func(int, panel_ignore_lid); \
+ func(int, semaphores); \
+ func(int, lvds_channel_mode); \
+ func(int, panel_use_ssc); \
+ func(int, vbt_sdvo_panel_type); \
+ func(int, enable_rc6); \
+ func(int, enable_dc); \
+ func(int, enable_fbc); \
+ func(int, enable_ppgtt); \
+ func(int, enable_execlists); \
+ func(int, enable_psr); \
+ func(int, disable_power_well); \
+ func(int, enable_ips); \
+ func(int, invert_brightness); \
+ func(int, enable_guc_loading); \
+ func(int, enable_guc_submission); \
+ func(int, guc_log_level); \
+ func(char *, guc_firmware_path); \
+ func(char *, huc_firmware_path); \
+ func(int, use_mmio_flip); \
+ func(int, mmio_debug); \
+ func(int, edp_vswing); \
+ func(unsigned int, inject_load_failure); \
+ /* leave bools at the end to not create holes */ \
+ func(bool, alpha_support); \
+ func(bool, enable_cmd_parser); \
+ func(bool, enable_hangcheck); \
+ func(bool, fastboot); \
+ func(bool, prefault_disable); \
+ func(bool, load_detect_test); \
+ func(bool, force_reset_modeset_test); \
+ func(bool, reset); \
+ func(bool, error_capture); \
+ func(bool, disable_display); \
+ func(bool, verbose_state_checks); \
+ func(bool, nuclear_pageflip); \
+ func(bool, enable_dp_mst); \
+ func(bool, enable_dpcd_backlight); \
+ func(bool, enable_gvt)
+
+#define MEMBER(T, member) T member
struct i915_params {
- int modeset;
- int panel_ignore_lid;
- int semaphores;
- int lvds_channel_mode;
- int panel_use_ssc;
- int vbt_sdvo_panel_type;
- int enable_rc6;
- int enable_dc;
- int enable_fbc;
- int enable_ppgtt;
- int enable_execlists;
- int enable_psr;
- unsigned int alpha_support;
- int disable_power_well;
- int enable_ips;
- int invert_brightness;
- int enable_guc_loading;
- int enable_guc_submission;
- int guc_log_level;
- int use_mmio_flip;
- int mmio_debug;
- int edp_vswing;
- unsigned int inject_load_failure;
- /* leave bools at the end to not create holes */
- bool enable_cmd_parser;
- bool enable_hangcheck;
- bool fastboot;
- bool prefault_disable;
- bool load_detect_test;
- bool force_reset_modeset_test;
- bool reset;
- bool error_capture;
- bool disable_display;
- bool verbose_state_checks;
- bool nuclear_pageflip;
- bool enable_dp_mst;
- bool enable_dpcd_backlight;
- bool enable_gvt;
+ I915_PARAMS_FOR_EACH(MEMBER);
};
+#undef MEMBER
extern struct i915_params i915 __read_mostly;
#include <linux/vga_switcheroo.h>
#include "i915_drv.h"
+#include "i915_selftest.h"
#define GEN_DEFAULT_PIPEOFFSETS \
.pipe_offsets = { PIPE_A_OFFSET, PIPE_B_OFFSET, \
.platform = INTEL_GEMINILAKE,
.is_alpha_support = 1,
.ddb_size = 1024,
+ .color = { .degamma_lut_size = 0, .gamma_lut_size = 1024 }
};
static const struct intel_device_info intel_kabylake_info = {
};
MODULE_DEVICE_TABLE(pci, pciidlist);
+static void i915_pci_remove(struct pci_dev *pdev)
+{
+ struct drm_device *dev = pci_get_drvdata(pdev);
+
+ i915_driver_unload(dev);
+ drm_dev_unref(dev);
+}
+
static int i915_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
{
struct intel_device_info *intel_info =
(struct intel_device_info *) ent->driver_data;
+ int err;
if (IS_ALPHA_SUPPORT(intel_info) && !i915.alpha_support) {
DRM_INFO("The driver support for your hardware in this kernel version is alpha quality\n"
if (vga_switcheroo_client_probe_defer(pdev))
return -EPROBE_DEFER;
- return i915_driver_load(pdev, ent);
-}
+ err = i915_driver_load(pdev, ent);
+ if (err)
+ return err;
-static void i915_pci_remove(struct pci_dev *pdev)
-{
- struct drm_device *dev = pci_get_drvdata(pdev);
+ err = i915_live_selftests(pdev);
+ if (err) {
+ i915_pci_remove(pdev);
+ return err > 0 ? -ENOTTY : err;
+ }
- i915_driver_unload(dev);
- drm_dev_unref(dev);
+ return 0;
}
static struct pci_driver i915_pci_driver = {
static int __init i915_init(void)
{
bool use_kms = true;
+ int err;
+
+ err = i915_mock_selftests();
+ if (err)
+ return err > 0 ? 0 : err;
/*
* Enable KMS by default, unless explicitly overriden by
static void gen7_update_oacontrol_locked(struct drm_i915_private *dev_priv)
{
- assert_spin_locked(&dev_priv->perf.hook_lock);
+ lockdep_assert_held(&dev_priv->perf.hook_lock);
if (dev_priv->perf.oa.exclusive_stream->enabled) {
struct i915_gem_context *ctx =
return !i915_mmio_reg_equal(reg, INVALID_MMIO_REG);
}
+#define _PICK(__index, ...) (((const u32 []){ __VA_ARGS__ })[__index])
+
#define _PIPE(pipe, a, b) ((a) + (pipe)*((b)-(a)))
#define _MMIO_PIPE(pipe, a, b) _MMIO(_PIPE(pipe, a, b))
#define _PLANE(plane, a, b) _PIPE(plane, a, b)
#define _MMIO_TRANS(tran, a, b) _MMIO(_TRANS(tran, a, b))
#define _PORT(port, a, b) ((a) + (port)*((b)-(a)))
#define _MMIO_PORT(port, a, b) _MMIO(_PORT(port, a, b))
-#define _PIPE3(pipe, a, b, c) ((pipe) == PIPE_A ? (a) : \
- (pipe) == PIPE_B ? (b) : (c))
+#define _PIPE3(pipe, ...) _PICK(pipe, __VA_ARGS__)
#define _MMIO_PIPE3(pipe, a, b, c) _MMIO(_PIPE3(pipe, a, b, c))
-#define _PORT3(port, a, b, c) ((port) == PORT_A ? (a) : \
- (port) == PORT_B ? (b) : (c))
+#define _PORT3(port, ...) _PICK(port, __VA_ARGS__)
#define _MMIO_PORT3(pipe, a, b, c) _MMIO(_PORT3(pipe, a, b, c))
-#define _PHY3(phy, a, b, c) ((phy) == DPIO_PHY0 ? (a) : \
- (phy) == DPIO_PHY1 ? (b) : (c))
+#define _PHY3(phy, ...) _PICK(phy, __VA_ARGS__)
#define _MMIO_PHY3(phy, a, b, c) _MMIO(_PHY3(phy, a, b, c))
#define _MASKED_FIELD(mask, value) ({ \
#define _MASKED_BIT_ENABLE(a) ({ typeof(a) _a = (a); _MASKED_FIELD(_a, _a); })
#define _MASKED_BIT_DISABLE(a) (_MASKED_FIELD((a), 0))
+/* Engine ID */
+#define RCS_HW 0
+#define VCS_HW 1
+#define BCS_HW 2
+#define VECS_HW 3
+#define VCS2_HW 4
/* PCI config space */
#define GCFGC 0xf0 /* 915+ only */
#define GC_LOW_FREQUENCY_ENABLE (1 << 7)
#define GC_DISPLAY_CLOCK_190_200_MHZ (0 << 4)
-#define GC_DISPLAY_CLOCK_333_MHZ (4 << 4)
+#define GC_DISPLAY_CLOCK_333_320_MHZ (4 << 4)
#define GC_DISPLAY_CLOCK_267_MHZ_PNV (0 << 4)
#define GC_DISPLAY_CLOCK_333_MHZ_PNV (1 << 4)
#define GC_DISPLAY_CLOCK_444_MHZ_PNV (2 << 4)
#define VLV_BIAS_CPU_125_SOC_875 (6 << 2)
#define CHV_BIAS_CPU_50_SOC_50 (3 << 2)
-#define VLV_CZ_CLOCK_TO_MILLI_SEC 100000
-
/* vlv2 north clock has */
#define CCK_FUSE_REG 0x8
#define CCK_FUSE_HPLL_FREQ_MASK 0x3
_MMIO(_BXT_PHY_CH(phy, ch, reg_ch0, reg_ch1))
#define BXT_P_CR_GT_DISP_PWRON _MMIO(0x138090)
+#define MIPIO_RST_CTRL (1 << 2)
#define _BXT_PHY_CTL_DDI_A 0x64C00
#define _BXT_PHY_CTL_DDI_B 0x64C10
INTEL_LEGACY_64B_CONTEXT
};
+enum {
+ FAULT_AND_HANG = 0,
+ FAULT_AND_HALT, /* Debug only */
+ FAULT_AND_STREAM,
+ FAULT_AND_CONTINUE /* Unsupported */
+};
+
+#define GEN8_CTX_VALID (1<<0)
+#define GEN8_CTX_FORCE_PD_RESTORE (1<<1)
+#define GEN8_CTX_FORCE_RESTORE (1<<2)
+#define GEN8_CTX_L3LLC_COHERENT (1<<5)
+#define GEN8_CTX_PRIVILEGE (1<<8)
#define GEN8_CTX_ADDRESSING_MODE_SHIFT 3
-#define GEN8_CTX_ADDRESSING_MODE(dev_priv) (USES_FULL_48BIT_PPGTT(dev_priv) ?\
- INTEL_LEGACY_64B_CONTEXT : \
- INTEL_LEGACY_32B_CONTEXT)
+
+#define GEN8_CTX_ID_SHIFT 32
+#define GEN8_CTX_ID_WIDTH 21
#define CHV_CLK_CTL1 _MMIO(0x101100)
#define VLV_CLK_CTL2 _MMIO(0x101104)
#define _PLANE_KEYMSK_2_A 0x70298
#define _PLANE_KEYMAX_1_A 0x701a0
#define _PLANE_KEYMAX_2_A 0x702a0
+#define _PLANE_COLOR_CTL_1_A 0x701CC /* GLK+ */
+#define _PLANE_COLOR_CTL_2_A 0x702CC /* GLK+ */
+#define _PLANE_COLOR_CTL_3_A 0x703CC /* GLK+ */
+#define PLANE_COLOR_PIPE_GAMMA_ENABLE (1 << 30)
+#define PLANE_COLOR_PIPE_CSC_ENABLE (1 << 23)
+#define PLANE_COLOR_PLANE_GAMMA_DISABLE (1 << 13)
#define _PLANE_BUF_CFG_1_A 0x7027c
#define _PLANE_BUF_CFG_2_A 0x7037c
#define _PLANE_NV12_BUF_CFG_1_A 0x70278
#define _PLANE_NV12_BUF_CFG_2_A 0x70378
+
#define _PLANE_CTL_1_B 0x71180
#define _PLANE_CTL_2_B 0x71280
#define _PLANE_CTL_3_B 0x71380
#define PLANE_NV12_BUF_CFG(pipe, plane) \
_MMIO_PLANE(plane, _PLANE_NV12_BUF_CFG_1(pipe), _PLANE_NV12_BUF_CFG_2(pipe))
-/* SKL new cursor registers */
+#define _PLANE_COLOR_CTL_1_B 0x711CC
+#define _PLANE_COLOR_CTL_2_B 0x712CC
+#define _PLANE_COLOR_CTL_3_B 0x713CC
+#define _PLANE_COLOR_CTL_1(pipe) \
+ _PIPE(pipe, _PLANE_COLOR_CTL_1_A, _PLANE_COLOR_CTL_1_B)
+#define _PLANE_COLOR_CTL_2(pipe) \
+ _PIPE(pipe, _PLANE_COLOR_CTL_2_A, _PLANE_COLOR_CTL_2_B)
+#define PLANE_COLOR_CTL(pipe, plane) \
+ _MMIO_PLANE(plane, _PLANE_COLOR_CTL_1(pipe), _PLANE_COLOR_CTL_2(pipe))
+
+#/* SKL new cursor registers */
#define _CUR_BUF_CFG_A 0x7017c
#define _CUR_BUF_CFG_B 0x7117c
#define CUR_BUF_CFG(pipe) _MMIO_PIPE(pipe, _CUR_BUF_CFG_A, _CUR_BUF_CFG_B)
#define CHICKEN_PAR2_1 _MMIO(0x42090)
#define KVM_CONFIG_CHANGE_NOTIFICATION_SELECT (1 << 14)
+#define CHICKEN_MISC_2 _MMIO(0x42084)
+#define GLK_CL0_PWR_DOWN (1 << 10)
+#define GLK_CL1_PWR_DOWN (1 << 11)
+#define GLK_CL2_PWR_DOWN (1 << 12)
+
#define _CHICKEN_PIPESL_1_A 0x420b0
#define _CHICKEN_PIPESL_1_B 0x420b4
#define HSW_FBCQ_DIS (1 << 22)
#define VLV_RCEDATA _MMIO(0xA0BC)
#define GEN6_RC6pp_THRESHOLD _MMIO(0xA0C0)
#define GEN6_PMINTRMSK _MMIO(0xA168)
-#define GEN8_PMINTR_REDIRECT_TO_GUC (1<<31)
+#define GEN8_PMINTR_DISABLE_REDIRECT_TO_GUC (1<<31)
+#define ARAT_EXPIRED_INTRMSK (1<<9)
#define GEN8_MISC_CTRL0 _MMIO(0xA180)
#define VLV_PWRDWNUPCTL _MMIO(0xA294)
#define GEN9_MEDIA_PG_IDLE_HYSTERESIS _MMIO(0xA0C4)
#define PAL_PREC_10_12_BIT (0 << 31)
#define PAL_PREC_SPLIT_MODE (1 << 31)
#define PAL_PREC_AUTO_INCREMENT (1 << 15)
+#define PAL_PREC_INDEX_VALUE_MASK (0x3ff << 0)
#define _PAL_PREC_DATA_A 0x4A404
#define _PAL_PREC_DATA_B 0x4AC04
#define _PAL_PREC_DATA_C 0x4B404
#define _PAL_PREC_EXT_GC_MAX_A 0x4A420
#define _PAL_PREC_EXT_GC_MAX_B 0x4AC20
#define _PAL_PREC_EXT_GC_MAX_C 0x4B420
+#define _PAL_PREC_EXT2_GC_MAX_A 0x4A430
+#define _PAL_PREC_EXT2_GC_MAX_B 0x4AC30
+#define _PAL_PREC_EXT2_GC_MAX_C 0x4B430
#define PREC_PAL_INDEX(pipe) _MMIO_PIPE(pipe, _PAL_PREC_INDEX_A, _PAL_PREC_INDEX_B)
#define PREC_PAL_DATA(pipe) _MMIO_PIPE(pipe, _PAL_PREC_DATA_A, _PAL_PREC_DATA_B)
#define PREC_PAL_GC_MAX(pipe, i) _MMIO(_PIPE(pipe, _PAL_PREC_GC_MAX_A, _PAL_PREC_GC_MAX_B) + (i) * 4)
#define PREC_PAL_EXT_GC_MAX(pipe, i) _MMIO(_PIPE(pipe, _PAL_PREC_EXT_GC_MAX_A, _PAL_PREC_EXT_GC_MAX_B) + (i) * 4)
+#define _PRE_CSC_GAMC_INDEX_A 0x4A484
+#define _PRE_CSC_GAMC_INDEX_B 0x4AC84
+#define _PRE_CSC_GAMC_INDEX_C 0x4B484
+#define PRE_CSC_GAMC_AUTO_INCREMENT (1 << 10)
+#define _PRE_CSC_GAMC_DATA_A 0x4A488
+#define _PRE_CSC_GAMC_DATA_B 0x4AC88
+#define _PRE_CSC_GAMC_DATA_C 0x4B488
+
+#define PRE_CSC_GAMC_INDEX(pipe) _MMIO_PIPE(pipe, _PRE_CSC_GAMC_INDEX_A, _PRE_CSC_GAMC_INDEX_B)
+#define PRE_CSC_GAMC_DATA(pipe) _MMIO_PIPE(pipe, _PRE_CSC_GAMC_DATA_A, _PRE_CSC_GAMC_DATA_B)
+
/* pipe CSC & degamma/gamma LUTs on CHV */
#define _CGM_PIPE_A_CSC_COEFF01 (VLV_DISPLAY_BASE + 0x67900)
#define _CGM_PIPE_A_CSC_COEFF23 (VLV_DISPLAY_BASE + 0x67904)
/* MIPI DSI registers */
-#define _MIPI_PORT(port, a, c) _PORT3(port, a, 0, c) /* ports A and C only */
+#define _MIPI_PORT(port, a, c) ((port) ? c : a) /* ports A and C only */
#define _MMIO_MIPI(port, a, c) _MMIO(_MIPI_PORT(port, a, c))
+#define MIPIO_TXESC_CLK_DIV1 _MMIO(0x160004)
+#define GLK_TX_ESC_CLK_DIV1_MASK 0x3FF
+#define MIPIO_TXESC_CLK_DIV2 _MMIO(0x160008)
+#define GLK_TX_ESC_CLK_DIV2_MASK 0x3FF
+
/* BXT MIPI clock controls */
#define BXT_MAX_VAR_OUTPUT_KHZ 39500
#define BXT_DSI_PLL_PVD_RATIO_SHIFT 16
#define BXT_DSI_PLL_PVD_RATIO_MASK (3 << BXT_DSI_PLL_PVD_RATIO_SHIFT)
#define BXT_DSI_PLL_PVD_RATIO_1 (1 << BXT_DSI_PLL_PVD_RATIO_SHIFT)
+#define BXT_DSIC_16X_BY1 (0 << 10)
#define BXT_DSIC_16X_BY2 (1 << 10)
#define BXT_DSIC_16X_BY3 (2 << 10)
#define BXT_DSIC_16X_BY4 (3 << 10)
#define BXT_DSIC_16X_MASK (3 << 10)
+#define BXT_DSIA_16X_BY1 (0 << 8)
#define BXT_DSIA_16X_BY2 (1 << 8)
#define BXT_DSIA_16X_BY3 (2 << 8)
#define BXT_DSIA_16X_BY4 (3 << 8)
#define BXT_DSI_PLL_RATIO_MAX 0x7D
#define BXT_DSI_PLL_RATIO_MIN 0x22
+#define GLK_DSI_PLL_RATIO_MAX 0x6F
+#define GLK_DSI_PLL_RATIO_MIN 0x22
#define BXT_DSI_PLL_RATIO_MASK 0xFF
#define BXT_REF_CLOCK_KHZ 19200
#define _BXT_MIPIC_PORT_CTRL 0x6B8C0
#define BXT_MIPI_PORT_CTRL(tc) _MMIO_MIPI(tc, _BXT_MIPIA_PORT_CTRL, _BXT_MIPIC_PORT_CTRL)
+#define BXT_P_DSI_REGULATOR_CFG _MMIO(0x160020)
+#define STAP_SELECT (1 << 0)
+
+#define BXT_P_DSI_REGULATOR_TX_CTRL _MMIO(0x160054)
+#define HS_IO_CTRL_SELECT (1 << 0)
+
#define DPI_ENABLE (1 << 31) /* A + C */
#define MIPIA_MIPI4DPHY_DELAY_COUNT_SHIFT 27
#define MIPIA_MIPI4DPHY_DELAY_COUNT_MASK (0xf << 27)
#define LP_BYTECLK_SHIFT 0
#define LP_BYTECLK_MASK (0xffff << 0)
+#define _MIPIA_TLPX_TIME_COUNT (dev_priv->mipi_mmio_base + 0xb0a4)
+#define _MIPIC_TLPX_TIME_COUNT (dev_priv->mipi_mmio_base + 0xb8a4)
+#define MIPI_TLPX_TIME_COUNT(port) _MMIO_MIPI(port, _MIPIA_TLPX_TIME_COUNT, _MIPIC_TLPX_TIME_COUNT)
+
+#define _MIPIA_CLK_LANE_TIMING (dev_priv->mipi_mmio_base + 0xb098)
+#define _MIPIC_CLK_LANE_TIMING (dev_priv->mipi_mmio_base + 0xb898)
+#define MIPI_CLK_LANE_TIMING(port) _MMIO_MIPI(port, _MIPIA_CLK_LANE_TIMING, _MIPIC_CLK_LANE_TIMING)
+
/* bits 31:0 */
#define _MIPIA_LP_GEN_DATA (dev_priv->mipi_mmio_base + 0xb064)
#define _MIPIC_LP_GEN_DATA (dev_priv->mipi_mmio_base + 0xb864)
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#ifndef __I915_SELFTEST_H__
+#define __I915_SELFTEST_H__
+
+struct pci_dev;
+struct drm_i915_private;
+
+struct i915_selftest {
+ unsigned long timeout_jiffies;
+ unsigned int timeout_ms;
+ unsigned int random_seed;
+ int mock;
+ int live;
+};
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include <linux/fault-inject.h>
+
+extern struct i915_selftest i915_selftest;
+
+int i915_mock_selftests(void);
+int i915_live_selftests(struct pci_dev *pdev);
+
+/* We extract the function declarations from i915_mock_selftests.h and
+ * i915_live_selftests.h Add your unit test declarations there!
+ *
+ * Mock unit tests are run very early upon module load, before the driver
+ * is probed. All hardware interactions, as well as other subsystems, must
+ * be "mocked".
+ *
+ * Live unit tests are run after the driver is loaded - all hardware
+ * interactions are real.
+ */
+#define selftest(name, func) int func(void);
+#include "selftests/i915_mock_selftests.h"
+#undef selftest
+#define selftest(name, func) int func(struct drm_i915_private *i915);
+#include "selftests/i915_live_selftests.h"
+#undef selftest
+
+struct i915_subtest {
+ int (*func)(void *data);
+ const char *name;
+};
+
+int __i915_subtests(const char *caller,
+ const struct i915_subtest *st,
+ unsigned int count,
+ void *data);
+#define i915_subtests(T, data) \
+ __i915_subtests(__func__, T, ARRAY_SIZE(T), data)
+
+#define SUBTEST(x) { x, #x }
+
+#define I915_SELFTEST_DECLARE(x) x
+#define I915_SELFTEST_ONLY(x) unlikely(x)
+
+#else /* !IS_ENABLED(CONFIG_DRM_I915_SELFTEST) */
+
+static inline int i915_mock_selftests(void) { return 0; }
+static inline int i915_live_selftests(struct pci_dev *pdev) { return 0; }
+
+#define I915_SELFTEST_DECLARE(x)
+#define I915_SELFTEST_ONLY(x) 0
+
+#endif
+
+/* Using the i915_selftest_ prefix becomes a little unwieldy with the helpers.
+ * Instead we use the igt_ shorthand, in reference to the intel-gpu-tools
+ * suite of uabi test cases (which includes a test runner for our selftests).
+ */
+
+#define IGT_TIMEOUT(name__) \
+ unsigned long name__ = jiffies + i915_selftest.timeout_jiffies
+
+__printf(2, 3)
+bool __igt_timeout(unsigned long timeout, const char *fmt, ...);
+
+#define igt_timeout(t, fmt, ...) \
+ __igt_timeout((t), KERN_WARNING pr_fmt(fmt), ##__VA_ARGS__)
+
+#define igt_can_mi_store_dword_imm(D) (INTEL_GEN(D) > 2)
+
+#endif /* !__I915_SELFTEST_H__ */
{
struct i915_sw_dma_fence_cb *cb = (struct i915_sw_dma_fence_cb *)data;
- printk(KERN_WARNING "asynchronous wait on fence %s:%s:%x timed out\n",
- cb->dma->ops->get_driver_name(cb->dma),
- cb->dma->ops->get_timeline_name(cb->dma),
- cb->dma->seqno);
+ pr_warn("asynchronous wait on fence %s:%s:%x timed out\n",
+ cb->dma->ops->get_driver_name(cb->dma),
+ cb->dma->ops->get_timeline_name(cb->dma),
+ cb->dma->seqno);
dma_fence_put(cb->dma);
cb->dma = NULL;
static u32 calc_residency(struct drm_i915_private *dev_priv,
i915_reg_t reg)
{
- u64 raw_time; /* 32b value may overflow during fixed point math */
- u64 units = 128ULL, div = 100000ULL;
- u32 ret;
-
- if (!intel_enable_rc6())
- return 0;
-
- intel_runtime_pm_get(dev_priv);
-
- /* On VLV and CHV, residency time is in CZ units rather than 1.28us */
- if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
- units = 1;
- div = dev_priv->czclk_freq;
-
- if (I915_READ(VLV_COUNTER_CONTROL) & VLV_COUNT_RANGE_HIGH)
- units <<= 8;
- } else if (IS_GEN9_LP(dev_priv)) {
- units = 1;
- div = 1200; /* 833.33ns */
- }
-
- raw_time = I915_READ(reg) * units;
- ret = DIV_ROUND_UP_ULL(raw_time, div);
-
- intel_runtime_pm_put(dev_priv);
- return ret;
+ return DIV_ROUND_CLOSEST_ULL(intel_rc6_residency_us(dev_priv, reg),
+ 1000);
}
static ssize_t
/* We still need *_set_rps to process the new max_delay and
* update the interrupt limits and PMINTRMSK even though
* frequency request may be unchanged. */
- intel_set_rps(dev_priv, val);
+ ret = intel_set_rps(dev_priv, val);
mutex_unlock(&dev_priv->rps.hw_lock);
intel_runtime_pm_put(dev_priv);
- return count;
+ return ret ?: count;
}
static ssize_t gt_min_freq_mhz_show(struct device *kdev, struct device_attribute *attr, char *buf)
/* We still need *_set_rps to process the new min_delay and
* update the interrupt limits and PMINTRMSK even though
* frequency request may be unchanged. */
- intel_set_rps(dev_priv, val);
+ ret = intel_set_rps(dev_priv, val);
mutex_unlock(&dev_priv->rps.hw_lock);
intel_runtime_pm_put(dev_priv);
- return count;
-
+ return ret ?: count;
}
static DEVICE_ATTR(gt_act_freq_mhz, S_IRUGO, gt_act_freq_mhz_show, NULL);
struct device *kdev = kobj_to_dev(kobj);
struct drm_i915_private *dev_priv = kdev_minor_to_i915(kdev);
- struct drm_device *dev = &dev_priv->drm;
- struct i915_error_state_file_priv error_priv;
struct drm_i915_error_state_buf error_str;
- ssize_t ret_count = 0;
- int ret;
-
- memset(&error_priv, 0, sizeof(error_priv));
+ struct i915_gpu_state *gpu;
+ ssize_t ret;
- ret = i915_error_state_buf_init(&error_str, to_i915(dev), count, off);
+ ret = i915_error_state_buf_init(&error_str, dev_priv, count, off);
if (ret)
return ret;
- error_priv.i915 = dev_priv;
- i915_error_state_get(dev, &error_priv);
-
- ret = i915_error_state_to_str(&error_str, &error_priv);
+ gpu = i915_first_error_state(dev_priv);
+ ret = i915_error_state_to_str(&error_str, gpu);
if (ret)
goto out;
- ret_count = count < error_str.bytes ? count : error_str.bytes;
+ ret = count < error_str.bytes ? count : error_str.bytes;
+ memcpy(buf, error_str.buf, ret);
- memcpy(buf, error_str.buf, ret_count);
out:
- i915_error_state_put(&error_priv);
+ i915_gpu_state_put(gpu);
i915_error_state_buf_release(&error_str);
- return ret ?: ret_count;
+ return ret;
}
static ssize_t error_state_write(struct file *file, struct kobject *kobj,
struct drm_i915_private *dev_priv = kdev_minor_to_i915(kdev);
DRM_DEBUG_DRIVER("Resetting error state\n");
- i915_destroy_error_state(dev_priv);
+ i915_reset_error_state(dev_priv);
return count;
}
#define TRACE_SYSTEM i915
#define TRACE_INCLUDE_FILE i915_trace
+/* watermark/fifo updates */
+
+TRACE_EVENT(intel_cpu_fifo_underrun,
+ TP_PROTO(struct drm_i915_private *dev_priv, enum pipe pipe),
+ TP_ARGS(dev_priv, pipe),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = pipe;
+ __entry->frame = dev_priv->drm.driver->get_vblank_counter(&dev_priv->drm, pipe);
+ __entry->scanline = intel_get_crtc_scanline(intel_get_crtc_for_pipe(dev_priv, pipe));
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline)
+);
+
+TRACE_EVENT(intel_pch_fifo_underrun,
+ TP_PROTO(struct drm_i915_private *dev_priv, enum transcoder pch_transcoder),
+ TP_ARGS(dev_priv, pch_transcoder),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ ),
+
+ TP_fast_assign(
+ enum pipe pipe = (enum pipe)pch_transcoder;
+ __entry->pipe = pipe;
+ __entry->frame = dev_priv->drm.driver->get_vblank_counter(&dev_priv->drm, pipe);
+ __entry->scanline = intel_get_crtc_scanline(intel_get_crtc_for_pipe(dev_priv, pipe));
+ ),
+
+ TP_printk("pch transcoder %c, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline)
+);
+
+TRACE_EVENT(intel_memory_cxsr,
+ TP_PROTO(struct drm_i915_private *dev_priv, bool old, bool new),
+ TP_ARGS(dev_priv, old, new),
+
+ TP_STRUCT__entry(
+ __array(u32, frame, 3)
+ __array(u32, scanline, 3)
+ __field(bool, old)
+ __field(bool, new)
+ ),
+
+ TP_fast_assign(
+ enum pipe pipe;
+ for_each_pipe(dev_priv, pipe) {
+ __entry->frame[pipe] =
+ dev_priv->drm.driver->get_vblank_counter(&dev_priv->drm, pipe);
+ __entry->scanline[pipe] =
+ intel_get_crtc_scanline(intel_get_crtc_for_pipe(dev_priv, pipe));
+ }
+ __entry->old = old;
+ __entry->new = new;
+ ),
+
+ TP_printk("%s->%s, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
+ onoff(__entry->old), onoff(__entry->new),
+ __entry->frame[PIPE_A], __entry->scanline[PIPE_A],
+ __entry->frame[PIPE_B], __entry->scanline[PIPE_B],
+ __entry->frame[PIPE_C], __entry->scanline[PIPE_C])
+);
+
+TRACE_EVENT(vlv_wm,
+ TP_PROTO(struct intel_crtc *crtc, const struct vlv_wm_values *wm),
+ TP_ARGS(crtc, wm),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __field(u32, level)
+ __field(u32, cxsr)
+ __field(u32, primary)
+ __field(u32, sprite0)
+ __field(u32, sprite1)
+ __field(u32, cursor)
+ __field(u32, sr_plane)
+ __field(u32, sr_cursor)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->frame = crtc->base.dev->driver->get_vblank_counter(crtc->base.dev,
+ crtc->pipe);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ __entry->level = wm->level;
+ __entry->cxsr = wm->cxsr;
+ __entry->primary = wm->pipe[crtc->pipe].plane[PLANE_PRIMARY];
+ __entry->sprite0 = wm->pipe[crtc->pipe].plane[PLANE_SPRITE0];
+ __entry->sprite1 = wm->pipe[crtc->pipe].plane[PLANE_SPRITE1];
+ __entry->cursor = wm->pipe[crtc->pipe].plane[PLANE_CURSOR];
+ __entry->sr_plane = wm->sr.plane;
+ __entry->sr_cursor = wm->sr.cursor;
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u, level=%d, cxsr=%d, wm %d/%d/%d/%d, sr %d/%d",
+ pipe_name(__entry->pipe), __entry->frame,
+ __entry->scanline, __entry->level, __entry->cxsr,
+ __entry->primary, __entry->sprite0, __entry->sprite1, __entry->cursor,
+ __entry->sr_plane, __entry->sr_cursor)
+);
+
+TRACE_EVENT(vlv_fifo_size,
+ TP_PROTO(struct intel_crtc *crtc, u32 sprite0_start, u32 sprite1_start, u32 fifo_size),
+ TP_ARGS(crtc, sprite0_start, sprite1_start, fifo_size),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __field(u32, sprite0_start)
+ __field(u32, sprite1_start)
+ __field(u32, fifo_size)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->frame = crtc->base.dev->driver->get_vblank_counter(crtc->base.dev,
+ crtc->pipe);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ __entry->sprite0_start = sprite0_start;
+ __entry->sprite1_start = sprite1_start;
+ __entry->fifo_size = fifo_size;
+ ),
+
+ TP_printk("pipe %c, frame=%u, scanline=%u, %d/%d/%d",
+ pipe_name(__entry->pipe), __entry->frame,
+ __entry->scanline, __entry->sprite0_start,
+ __entry->sprite1_start, __entry->fifo_size)
+);
+
+/* plane updates */
+
+TRACE_EVENT(intel_update_plane,
+ TP_PROTO(struct drm_plane *plane, struct intel_crtc *crtc),
+ TP_ARGS(plane, crtc),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(const char *, name)
+ __field(u32, frame)
+ __field(u32, scanline)
+ __array(int, src, 4)
+ __array(int, dst, 4)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->name = plane->name;
+ __entry->frame = crtc->base.dev->driver->get_vblank_counter(crtc->base.dev,
+ crtc->pipe);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ memcpy(__entry->src, &plane->state->src, sizeof(__entry->src));
+ memcpy(__entry->dst, &plane->state->dst, sizeof(__entry->dst));
+ ),
+
+ TP_printk("pipe %c, plane %s, frame=%u, scanline=%u, " DRM_RECT_FP_FMT " -> " DRM_RECT_FMT,
+ pipe_name(__entry->pipe), __entry->name,
+ __entry->frame, __entry->scanline,
+ DRM_RECT_FP_ARG((const struct drm_rect *)__entry->src),
+ DRM_RECT_ARG((const struct drm_rect *)__entry->dst))
+);
+
+TRACE_EVENT(intel_disable_plane,
+ TP_PROTO(struct drm_plane *plane, struct intel_crtc *crtc),
+ TP_ARGS(plane, crtc),
+
+ TP_STRUCT__entry(
+ __field(enum pipe, pipe)
+ __field(const char *, name)
+ __field(u32, frame)
+ __field(u32, scanline)
+ ),
+
+ TP_fast_assign(
+ __entry->pipe = crtc->pipe;
+ __entry->name = plane->name;
+ __entry->frame = crtc->base.dev->driver->get_vblank_counter(crtc->base.dev,
+ crtc->pipe);
+ __entry->scanline = intel_get_crtc_scanline(crtc);
+ ),
+
+ TP_printk("pipe %c, plane %s, frame=%u, scanline=%u",
+ pipe_name(__entry->pipe), __entry->name,
+ __entry->frame, __entry->scanline)
+);
+
/* pipe updates */
TRACE_EVENT(i915_pipe_update_start,
__entry->obj, __entry->offset, __entry->size, __entry->vm)
);
-TRACE_EVENT(i915_va_alloc,
- TP_PROTO(struct i915_vma *vma),
- TP_ARGS(vma),
-
- TP_STRUCT__entry(
- __field(struct i915_address_space *, vm)
- __field(u64, start)
- __field(u64, end)
- ),
-
- TP_fast_assign(
- __entry->vm = vma->vm;
- __entry->start = vma->node.start;
- __entry->end = vma->node.start + vma->node.size - 1;
- ),
-
- TP_printk("vm=%p (%c), 0x%llx-0x%llx",
- __entry->vm, i915_is_ggtt(__entry->vm) ? 'G' : 'P', __entry->start, __entry->end)
-);
-
-DECLARE_EVENT_CLASS(i915_px_entry,
- TP_PROTO(struct i915_address_space *vm, u32 px, u64 start, u64 px_shift),
- TP_ARGS(vm, px, start, px_shift),
-
- TP_STRUCT__entry(
- __field(struct i915_address_space *, vm)
- __field(u32, px)
- __field(u64, start)
- __field(u64, end)
- ),
-
- TP_fast_assign(
- __entry->vm = vm;
- __entry->px = px;
- __entry->start = start;
- __entry->end = ((start + (1ULL << px_shift)) & ~((1ULL << px_shift)-1)) - 1;
- ),
-
- TP_printk("vm=%p, pde=%d (0x%llx-0x%llx)",
- __entry->vm, __entry->px, __entry->start, __entry->end)
-);
-
-DEFINE_EVENT(i915_px_entry, i915_page_table_entry_alloc,
- TP_PROTO(struct i915_address_space *vm, u32 pde, u64 start, u64 pde_shift),
- TP_ARGS(vm, pde, start, pde_shift)
-);
-
-DEFINE_EVENT_PRINT(i915_px_entry, i915_page_directory_entry_alloc,
- TP_PROTO(struct i915_address_space *vm, u32 pdpe, u64 start, u64 pdpe_shift),
- TP_ARGS(vm, pdpe, start, pdpe_shift),
-
- TP_printk("vm=%p, pdpe=%d (0x%llx-0x%llx)",
- __entry->vm, __entry->px, __entry->start, __entry->end)
-);
-
-DEFINE_EVENT_PRINT(i915_px_entry, i915_page_directory_pointer_entry_alloc,
- TP_PROTO(struct i915_address_space *vm, u32 pml4e, u64 start, u64 pml4e_shift),
- TP_ARGS(vm, pml4e, start, pml4e_shift),
-
- TP_printk("vm=%p, pml4e=%d (0x%llx-0x%llx)",
- __entry->vm, __entry->px, __entry->start, __entry->end)
-);
-
-/* Avoid extra math because we only support two sizes. The format is defined by
- * bitmap_scnprintf. Each 32 bits is 8 HEX digits followed by comma */
-#define TRACE_PT_SIZE(bits) \
- ((((bits) == 1024) ? 288 : 144) + 1)
-
-DECLARE_EVENT_CLASS(i915_page_table_entry_update,
- TP_PROTO(struct i915_address_space *vm, u32 pde,
- struct i915_page_table *pt, u32 first, u32 count, u32 bits),
- TP_ARGS(vm, pde, pt, first, count, bits),
-
- TP_STRUCT__entry(
- __field(struct i915_address_space *, vm)
- __field(u32, pde)
- __field(u32, first)
- __field(u32, last)
- __dynamic_array(char, cur_ptes, TRACE_PT_SIZE(bits))
- ),
-
- TP_fast_assign(
- __entry->vm = vm;
- __entry->pde = pde;
- __entry->first = first;
- __entry->last = first + count - 1;
- scnprintf(__get_str(cur_ptes),
- TRACE_PT_SIZE(bits),
- "%*pb",
- bits,
- pt->used_ptes);
- ),
-
- TP_printk("vm=%p, pde=%d, updating %u:%u\t%s",
- __entry->vm, __entry->pde, __entry->last, __entry->first,
- __get_str(cur_ptes))
-);
-
-DEFINE_EVENT(i915_page_table_entry_update, i915_page_table_entry_map,
- TP_PROTO(struct i915_address_space *vm, u32 pde,
- struct i915_page_table *pt, u32 first, u32 count, u32 bits),
- TP_ARGS(vm, pde, pt, first, count, bits)
-);
-
-TRACE_EVENT(i915_gem_object_change_domain,
- TP_PROTO(struct drm_i915_gem_object *obj, u32 old_read, u32 old_write),
- TP_ARGS(obj, old_read, old_write),
-
- TP_STRUCT__entry(
- __field(struct drm_i915_gem_object *, obj)
- __field(u32, read_domains)
- __field(u32, write_domain)
- ),
-
- TP_fast_assign(
- __entry->obj = obj;
- __entry->read_domains = obj->base.read_domains | (old_read << 16);
- __entry->write_domain = obj->base.write_domain | (old_write << 16);
- ),
-
- TP_printk("obj=%p, read=%02x=>%02x, write=%02x=>%02x",
- __entry->obj,
- __entry->read_domains >> 16,
- __entry->read_domains & 0xffff,
- __entry->write_domain >> 16,
- __entry->write_domain & 0xffff)
-);
-
TRACE_EVENT(i915_gem_object_pwrite,
TP_PROTO(struct drm_i915_gem_object *obj, u32 offset, u32 len),
TP_ARGS(obj, offset, len),
__entry->seqno)
);
-TRACE_EVENT(i915_gem_ring_dispatch,
+TRACE_EVENT(i915_gem_request_queue,
TP_PROTO(struct drm_i915_gem_request *req, u32 flags),
TP_ARGS(req, flags),
TP_STRUCT__entry(
__field(u32, dev)
__field(u32, ring)
+ __field(u32, ctx)
__field(u32, seqno)
__field(u32, flags)
),
TP_fast_assign(
__entry->dev = req->i915->drm.primary->index;
__entry->ring = req->engine->id;
- __entry->seqno = req->global_seqno;
+ __entry->ctx = req->fence.context;
+ __entry->seqno = req->fence.seqno;
__entry->flags = flags;
- dma_fence_enable_sw_signaling(&req->fence);
),
- TP_printk("dev=%u, ring=%u, seqno=%u, flags=%x",
- __entry->dev, __entry->ring, __entry->seqno, __entry->flags)
+ TP_printk("dev=%u, ring=%u, ctx=%u, seqno=%u, flags=0x%x",
+ __entry->dev, __entry->ring, __entry->ctx, __entry->seqno,
+ __entry->flags)
);
TRACE_EVENT(i915_gem_ring_flush,
TP_STRUCT__entry(
__field(u32, dev)
+ __field(u32, ctx)
__field(u32, ring)
__field(u32, seqno)
+ __field(u32, global)
),
TP_fast_assign(
__entry->dev = req->i915->drm.primary->index;
__entry->ring = req->engine->id;
- __entry->seqno = req->global_seqno;
+ __entry->ctx = req->fence.context;
+ __entry->seqno = req->fence.seqno;
+ __entry->global = req->global_seqno;
),
- TP_printk("dev=%u, ring=%u, seqno=%u",
- __entry->dev, __entry->ring, __entry->seqno)
+ TP_printk("dev=%u, ring=%u, ctx=%u, seqno=%u, global=%u",
+ __entry->dev, __entry->ring, __entry->ctx, __entry->seqno,
+ __entry->global)
);
DEFINE_EVENT(i915_gem_request, i915_gem_request_add,
TP_ARGS(req)
);
-TRACE_EVENT(i915_gem_request_notify,
- TP_PROTO(struct intel_engine_cs *engine),
- TP_ARGS(engine),
+#if defined(CONFIG_DRM_I915_LOW_LEVEL_TRACEPOINTS)
+DEFINE_EVENT(i915_gem_request, i915_gem_request_submit,
+ TP_PROTO(struct drm_i915_gem_request *req),
+ TP_ARGS(req)
+);
+
+DEFINE_EVENT(i915_gem_request, i915_gem_request_execute,
+ TP_PROTO(struct drm_i915_gem_request *req),
+ TP_ARGS(req)
+);
+
+DECLARE_EVENT_CLASS(i915_gem_request_hw,
+ TP_PROTO(struct drm_i915_gem_request *req,
+ unsigned int port),
+ TP_ARGS(req, port),
+
+ TP_STRUCT__entry(
+ __field(u32, dev)
+ __field(u32, ring)
+ __field(u32, seqno)
+ __field(u32, global_seqno)
+ __field(u32, ctx)
+ __field(u32, port)
+ ),
+
+ TP_fast_assign(
+ __entry->dev = req->i915->drm.primary->index;
+ __entry->ring = req->engine->id;
+ __entry->ctx = req->fence.context;
+ __entry->seqno = req->fence.seqno;
+ __entry->global_seqno = req->global_seqno;
+ __entry->port = port;
+ ),
+
+ TP_printk("dev=%u, ring=%u, ctx=%u, seqno=%u, global=%u, port=%u",
+ __entry->dev, __entry->ring, __entry->ctx,
+ __entry->seqno, __entry->global_seqno,
+ __entry->port)
+);
+
+DEFINE_EVENT(i915_gem_request_hw, i915_gem_request_in,
+ TP_PROTO(struct drm_i915_gem_request *req, unsigned int port),
+ TP_ARGS(req, port)
+);
+
+DEFINE_EVENT(i915_gem_request, i915_gem_request_out,
+ TP_PROTO(struct drm_i915_gem_request *req),
+ TP_ARGS(req)
+);
+#else
+#if !defined(TRACE_HEADER_MULTI_READ)
+static inline void
+trace_i915_gem_request_submit(struct drm_i915_gem_request *req)
+{
+}
+
+static inline void
+trace_i915_gem_request_execute(struct drm_i915_gem_request *req)
+{
+}
+
+static inline void
+trace_i915_gem_request_in(struct drm_i915_gem_request *req, unsigned int port)
+{
+}
+
+static inline void
+trace_i915_gem_request_out(struct drm_i915_gem_request *req)
+{
+}
+#endif
+#endif
+
+TRACE_EVENT(intel_engine_notify,
+ TP_PROTO(struct intel_engine_cs *engine, bool waiters),
+ TP_ARGS(engine, waiters),
TP_STRUCT__entry(
__field(u32, dev)
__field(u32, ring)
__field(u32, seqno)
+ __field(bool, waiters)
),
TP_fast_assign(
__entry->dev = engine->i915->drm.primary->index;
__entry->ring = engine->id;
__entry->seqno = intel_engine_get_seqno(engine);
+ __entry->waiters = waiters;
),
- TP_printk("dev=%u, ring=%u, seqno=%u",
- __entry->dev, __entry->ring, __entry->seqno)
+ TP_printk("dev=%u, ring=%u, seqno=%u, waiters=%u",
+ __entry->dev, __entry->ring, __entry->seqno,
+ __entry->waiters)
);
DEFINE_EVENT(i915_gem_request, i915_gem_request_retire,
TP_ARGS(req)
);
-DEFINE_EVENT(i915_gem_request, i915_gem_request_complete,
- TP_PROTO(struct drm_i915_gem_request *req),
- TP_ARGS(req)
-);
-
TRACE_EVENT(i915_gem_request_wait_begin,
- TP_PROTO(struct drm_i915_gem_request *req),
- TP_ARGS(req),
+ TP_PROTO(struct drm_i915_gem_request *req, unsigned int flags),
+ TP_ARGS(req, flags),
TP_STRUCT__entry(
__field(u32, dev)
__field(u32, ring)
+ __field(u32, ctx)
__field(u32, seqno)
- __field(bool, blocking)
+ __field(u32, global)
+ __field(unsigned int, flags)
),
/* NB: the blocking information is racy since mutex_is_locked
TP_fast_assign(
__entry->dev = req->i915->drm.primary->index;
__entry->ring = req->engine->id;
- __entry->seqno = req->global_seqno;
- __entry->blocking =
- mutex_is_locked(&req->i915->drm.struct_mutex);
+ __entry->ctx = req->fence.context;
+ __entry->seqno = req->fence.seqno;
+ __entry->global = req->global_seqno;
+ __entry->flags = flags;
),
- TP_printk("dev=%u, ring=%u, seqno=%u, blocking=%s",
- __entry->dev, __entry->ring,
- __entry->seqno, __entry->blocking ? "yes (NB)" : "no")
+ TP_printk("dev=%u, ring=%u, ctx=%u, seqno=%u, global=%u, blocking=%u, flags=0x%x",
+ __entry->dev, __entry->ring, __entry->ctx, __entry->seqno,
+ __entry->global, !!(__entry->flags & I915_WAIT_LOCKED),
+ __entry->flags)
);
DEFINE_EVENT(i915_gem_request, i915_gem_request_wait_end,
TP_STRUCT__entry(
__field(u32, dev)
__field(struct i915_gem_context *, ctx)
+ __field(u32, hw_id)
__field(struct i915_address_space *, vm)
),
TP_fast_assign(
+ __entry->dev = ctx->i915->drm.primary->index;
__entry->ctx = ctx;
+ __entry->hw_id = ctx->hw_id;
__entry->vm = ctx->ppgtt ? &ctx->ppgtt->base : NULL;
- __entry->dev = ctx->i915->drm.primary->index;
),
- TP_printk("dev=%u, ctx=%p, ctx_vm=%p",
- __entry->dev, __entry->ctx, __entry->vm)
+ TP_printk("dev=%u, ctx=%p, ctx_vm=%p, hw_id=%u",
+ __entry->dev, __entry->ctx, __entry->vm, __entry->hw_id)
)
DEFINE_EVENT(i915_context, i915_context_create,
#ifndef __I915_UTILS_H
#define __I915_UTILS_H
+#if GCC_VERSION >= 70000
+#define add_overflows(A, B) \
+ __builtin_add_overflow_p((A), (B), (typeof((A) + (B)))0)
+#else
+#define add_overflows(A, B) ({ \
+ typeof(A) a = (A); \
+ typeof(B) b = (B); \
+ a + b < a; \
+})
+#endif
+
#define range_overflows(start, size, max) ({ \
typeof(start) start__ = (start); \
typeof(size) size__ = (size); \
#define ptr_pack_bits(ptr, bits) \
((typeof(ptr))((unsigned long)(ptr) | (bits)))
+#define ptr_offset(ptr, member) offsetof(typeof(*(ptr)), member)
+
#define fetch_and_zero(ptr) ({ \
typeof(*ptr) __T = *(ptr); \
*(ptr) = (typeof(*ptr))0; \
int intel_vgt_balloon(struct drm_i915_private *dev_priv)
{
struct i915_ggtt *ggtt = &dev_priv->ggtt;
- unsigned long ggtt_end = ggtt->base.start + ggtt->base.total;
+ unsigned long ggtt_end = ggtt->base.total;
unsigned long mappable_base, mappable_size, mappable_end;
unsigned long unmappable_base, unmappable_size, unmappable_end;
DRM_INFO("Unmappable graphic memory: base 0x%lx size %ldKiB\n",
unmappable_base, unmappable_size / 1024);
- if (mappable_base < ggtt->base.start ||
- mappable_end > ggtt->mappable_end ||
+ if (mappable_end > ggtt->mappable_end ||
unmappable_base < ggtt->mappable_end ||
unmappable_end > ggtt_end) {
DRM_ERROR("Invalid ballooning configuration!\n");
goto err;
}
- /*
- * No need to partition out the last physical page,
- * because it is reserved to the guard page.
- */
- if (unmappable_end < ggtt_end - PAGE_SIZE) {
+ if (unmappable_end < ggtt_end) {
ret = vgt_balloon_space(ggtt, &bl_info.space[3],
- unmappable_end, ggtt_end - PAGE_SIZE);
+ unmappable_end, ggtt_end);
if (ret)
goto err;
}
/* Mappable graphic memory ballooning */
- if (mappable_base > ggtt->base.start) {
+ if (mappable_base) {
ret = vgt_balloon_space(ggtt, &bl_info.space[0],
- ggtt->base.start, mappable_base);
+ 0, mappable_base);
if (ret)
goto err;
struct rb_node *rb, **p;
int i;
+ /* The aliasing_ppgtt should never be used directly! */
+ GEM_BUG_ON(vm == &vm->i915->mm.aliasing_ppgtt->base);
+
vma = kmem_cache_zalloc(vm->i915->vmas, GFP_KERNEL);
if (vma == NULL)
return ERR_PTR(-ENOMEM);
u32 vma_flags;
int ret;
- if (WARN_ON(flags == 0))
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
+ GEM_BUG_ON(vma->size > vma->node.size);
+
+ if (GEM_WARN_ON(range_overflows(vma->node.start,
+ vma->node.size,
+ vma->vm->total)))
+ return -ENODEV;
+
+ if (GEM_WARN_ON(!flags))
return -EINVAL;
bind_flags = 0;
if (bind_flags == 0)
return 0;
- if (GEM_WARN_ON(range_overflows(vma->node.start,
- vma->node.size,
- vma->vm->total)))
- return -ENODEV;
-
- if (vma_flags == 0 && vma->vm->allocate_va_range) {
- trace_i915_va_alloc(vma);
- ret = vma->vm->allocate_va_range(vma->vm,
- vma->node.start,
- vma->node.size);
- if (ret)
- return ret;
- }
-
trace_i915_vma_bind(vma, bind_flags);
ret = vma->vm->bind_vma(vma, cache_level, bind_flags);
if (ret)
__i915_gem_object_release_unless_active(obj);
}
-bool
-i915_vma_misplaced(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
+bool i915_vma_misplaced(const struct i915_vma *vma,
+ u64 size, u64 alignment, u64 flags)
{
if (!drm_mm_node_allocated(&vma->node))
return false;
return 0;
}
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/i915_vma.c"
+#endif
int i915_vma_bind(struct i915_vma *vma, enum i915_cache_level cache_level,
u32 flags);
bool i915_gem_valid_gtt_space(struct i915_vma *vma, unsigned long cache_level);
-bool
-i915_vma_misplaced(struct i915_vma *vma, u64 size, u64 alignment, u64 flags);
+bool i915_vma_misplaced(const struct i915_vma *vma,
+ u64 size, u64 alignment, u64 flags);
void __i915_vma_set_map_and_fenceable(struct i915_vma *vma);
int __must_check i915_vma_unbind(struct i915_vma *vma);
void i915_vma_close(struct i915_vma *vma);
crtc_state->update_wm_pre = false;
crtc_state->update_wm_post = false;
crtc_state->fb_changed = false;
+ crtc_state->fifo_changed = false;
crtc_state->wm.need_postvbl_update = false;
crtc_state->fb_bits = 0;
/**
* intel_atomic_setup_scalers() - setup scalers for crtc per staged requests
- * @dev: DRM device
+ * @dev_priv: i915 device
* @crtc: intel crtc
* @crtc_state: incoming crtc_state to validate and setup scalers
*
* 0 - scalers were setup succesfully
* error code - otherwise
*/
-int intel_atomic_setup_scalers(struct drm_device *dev,
- struct intel_crtc *intel_crtc,
- struct intel_crtc_state *crtc_state)
+int intel_atomic_setup_scalers(struct drm_i915_private *dev_priv,
+ struct intel_crtc *intel_crtc,
+ struct intel_crtc_state *crtc_state)
{
struct drm_plane *plane = NULL;
struct intel_plane *intel_plane;
*/
if (!plane) {
struct drm_plane_state *state;
- plane = drm_plane_from_index(dev, i);
+ plane = drm_plane_from_index(&dev_priv->drm, i);
state = drm_atomic_get_plane_state(drm_state, plane);
if (IS_ERR(state)) {
DRM_DEBUG_KMS("Failed to add [PLANE:%d] to drm_state\n",
}
/* set scaler mode */
- if (num_scalers_need == 1 && intel_crtc->pipe != PIPE_C) {
+ if (IS_GEMINILAKE(dev_priv)) {
+ scaler_state->scalers[*scaler_id].mode = 0;
+ } else if (num_scalers_need == 1 && intel_crtc->pipe != PIPE_C) {
/*
* when only 1 scaler is in use on either pipe A or B,
* scaler 0 operates in high quality (HQ) mode.
if (ret)
return ret;
+ /* FIXME pre-g4x don't work like this */
+ if (intel_state->base.visible)
+ crtc_state->active_planes |= BIT(intel_plane->id);
+ else
+ crtc_state->active_planes &= ~BIT(intel_plane->id);
+
return intel_plane_atomic_calc_changes(&crtc_state->base, state);
}
to_intel_plane_state(plane->state);
struct drm_crtc *crtc = plane->state->crtc ?: old_state->crtc;
- if (intel_state->base.visible)
+ if (intel_state->base.visible) {
+ trace_intel_update_plane(plane,
+ to_intel_crtc(crtc));
+
intel_plane->update_plane(plane,
to_intel_crtc_state(crtc->state),
intel_state);
- else
+ } else {
+ trace_intel_disable_plane(plane,
+ to_intel_crtc(crtc));
+
intel_plane->disable_plane(plane, crtc);
+ }
}
const struct drm_plane_helper_funcs intel_plane_helper_funcs = {
struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
u32 tmp;
- if (!IS_SKYLAKE(dev_priv) && !IS_KABYLAKE(dev_priv))
+ if (!IS_GEN9_BC(dev_priv))
return;
i915_audio_component_get_power(kdev);
if (WARN_ON_ONCE(!HAS_DDI(dev_priv)))
return -ENODEV;
- return dev_priv->cdclk_freq;
+ return dev_priv->cdclk.hw.cdclk;
}
/*
return;
}
+/* Common defaults which may be overridden by VBT. */
static void
init_vbt_defaults(struct drm_i915_private *dev_priv)
{
&dev_priv->vbt.ddi_port_info[port];
info->hdmi_level_shift = HDMI_LEVEL_SHIFT_UNKNOWN;
+ }
+}
+
+/* Defaults to initialize only if there is no VBT. */
+static void
+init_vbt_missing_defaults(struct drm_i915_private *dev_priv)
+{
+ enum port port;
+
+ for (port = PORT_A; port < I915_MAX_PORTS; port++) {
+ struct ddi_vbt_port_info *info =
+ &dev_priv->vbt.ddi_port_info[port];
info->supports_dvi = (port != PORT_A && port != PORT_E);
info->supports_hdmi = info->supports_dvi;
* intel_bios_init - find VBT and initialize settings from the BIOS
* @dev_priv: i915 device instance
*
- * Loads the Video BIOS and checks that the VBT exists. Sets scratch registers
- * to appropriate values.
- *
- * Returns 0 on success, nonzero on failure.
+ * Parse and initialize settings from the Video BIOS Tables (VBT). If the VBT
+ * was not found in ACPI OpRegion, try to find it in PCI ROM first. Also
+ * initialize some defaults if the VBT is not present at all.
*/
-int
-intel_bios_init(struct drm_i915_private *dev_priv)
+void intel_bios_init(struct drm_i915_private *dev_priv)
{
struct pci_dev *pdev = dev_priv->drm.pdev;
const struct vbt_header *vbt = dev_priv->opregion.vbt;
const struct bdb_header *bdb;
u8 __iomem *bios = NULL;
- if (HAS_PCH_NOP(dev_priv))
- return -ENODEV;
+ if (HAS_PCH_NOP(dev_priv)) {
+ DRM_DEBUG_KMS("Skipping VBT init due to disabled display.\n");
+ return;
+ }
init_vbt_defaults(dev_priv);
+ /* If the OpRegion does not have VBT, look in PCI ROM. */
if (!vbt) {
size_t size;
bios = pci_map_rom(pdev, &size);
if (!bios)
- return -1;
+ goto out;
vbt = find_vbt(bios, size);
- if (!vbt) {
- pci_unmap_rom(pdev, bios);
- return -1;
- }
+ if (!vbt)
+ goto out;
DRM_DEBUG_KMS("Found valid VBT in PCI ROM\n");
}
parse_mipi_sequence(dev_priv, bdb);
parse_ddi_ports(dev_priv, bdb);
+out:
+ if (!vbt) {
+ DRM_INFO("Failed to find VBIOS tables (VBT)\n");
+ init_vbt_missing_defaults(dev_priv);
+ }
+
if (bios)
pci_unmap_rom(pdev, bios);
-
- return 0;
}
/**
#include "i915_drv.h"
+static unsigned int __intel_breadcrumbs_wakeup(struct intel_breadcrumbs *b)
+{
+ struct intel_wait *wait;
+ unsigned int result = 0;
+
+ lockdep_assert_held(&b->irq_lock);
+
+ wait = b->irq_wait;
+ if (wait) {
+ result = ENGINE_WAKEUP_WAITER;
+ if (wake_up_process(wait->tsk))
+ result |= ENGINE_WAKEUP_ASLEEP;
+ }
+
+ return result;
+}
+
+unsigned int intel_engine_wakeup(struct intel_engine_cs *engine)
+{
+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
+ unsigned int result;
+
+ spin_lock_irq(&b->irq_lock);
+ result = __intel_breadcrumbs_wakeup(b);
+ spin_unlock_irq(&b->irq_lock);
+
+ return result;
+}
+
+static unsigned long wait_timeout(void)
+{
+ return round_jiffies_up(jiffies + DRM_I915_HANGCHECK_JIFFIES);
+}
+
+static noinline void missed_breadcrumb(struct intel_engine_cs *engine)
+{
+ DRM_DEBUG_DRIVER("%s missed breadcrumb at %pF, irq posted? %s\n",
+ engine->name, __builtin_return_address(0),
+ yesno(test_bit(ENGINE_IRQ_BREADCRUMB,
+ &engine->irq_posted)));
+
+ set_bit(engine->id, &engine->i915->gpu_error.missed_irq_rings);
+}
+
static void intel_breadcrumbs_hangcheck(unsigned long data)
{
struct intel_engine_cs *engine = (struct intel_engine_cs *)data;
struct intel_breadcrumbs *b = &engine->breadcrumbs;
- if (!b->irq_enabled)
+ if (!b->irq_armed)
return;
- if (time_before(jiffies, b->timeout)) {
- mod_timer(&b->hangcheck, b->timeout);
+ if (b->hangcheck_interrupts != atomic_read(&engine->irq_count)) {
+ b->hangcheck_interrupts = atomic_read(&engine->irq_count);
+ mod_timer(&b->hangcheck, wait_timeout());
return;
}
- DRM_DEBUG("Hangcheck timer elapsed... %s idle\n", engine->name);
- set_bit(engine->id, &engine->i915->gpu_error.missed_irq_rings);
- mod_timer(&engine->breadcrumbs.fake_irq, jiffies + 1);
+ /* We keep the hangcheck timer alive until we disarm the irq, even
+ * if there are no waiters at present.
+ *
+ * If the waiter was currently running, assume it hasn't had a chance
+ * to process the pending interrupt (e.g, low priority task on a loaded
+ * system) and wait until it sleeps before declaring a missed interrupt.
+ *
+ * If the waiter was asleep (and not even pending a wakeup), then we
+ * must have missed an interrupt as the GPU has stopped advancing
+ * but we still have a waiter. Assuming all batches complete within
+ * DRM_I915_HANGCHECK_JIFFIES [1.5s]!
+ */
+ if (intel_engine_wakeup(engine) & ENGINE_WAKEUP_ASLEEP) {
+ missed_breadcrumb(engine);
+ mod_timer(&engine->breadcrumbs.fake_irq, jiffies + 1);
+ } else {
+ mod_timer(&b->hangcheck, wait_timeout());
+ }
+}
+
+static void intel_breadcrumbs_fake_irq(unsigned long data)
+{
+ struct intel_engine_cs *engine = (struct intel_engine_cs *)data;
+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
+
+ /* The timer persists in case we cannot enable interrupts,
+ * or if we have previously seen seqno/interrupt incoherency
+ * ("missed interrupt" syndrome, better known as a "missed breadcrumb").
+ * Here the worker will wake up every jiffie in order to kick the
+ * oldest waiter to do the coherent seqno check.
+ */
+
+ spin_lock_irq(&b->irq_lock);
+ if (!__intel_breadcrumbs_wakeup(b))
+ __intel_engine_disarm_breadcrumbs(engine);
+ spin_unlock_irq(&b->irq_lock);
+ if (!b->irq_armed)
+ return;
+
+ mod_timer(&b->fake_irq, jiffies + 1);
/* Ensure that even if the GPU hangs, we get woken up.
*
i915_queue_hangcheck(engine->i915);
}
-static unsigned long wait_timeout(void)
-{
- return round_jiffies_up(jiffies + DRM_I915_HANGCHECK_JIFFIES);
-}
-
-static void intel_breadcrumbs_fake_irq(unsigned long data)
-{
- struct intel_engine_cs *engine = (struct intel_engine_cs *)data;
-
- /*
- * The timer persists in case we cannot enable interrupts,
- * or if we have previously seen seqno/interrupt incoherency
- * ("missed interrupt" syndrome). Here the worker will wake up
- * every jiffie in order to kick the oldest waiter to do the
- * coherent seqno check.
- */
- if (intel_engine_wakeup(engine))
- mod_timer(&engine->breadcrumbs.fake_irq, jiffies + 1);
-}
-
static void irq_enable(struct intel_engine_cs *engine)
{
/* Enabling the IRQ may miss the generation of the interrupt, but
* we still need to force the barrier before reading the seqno,
* just in case.
*/
- engine->breadcrumbs.irq_posted = true;
+ set_bit(ENGINE_IRQ_BREADCRUMB, &engine->irq_posted);
/* Caller disables interrupts */
spin_lock(&engine->i915->irq_lock);
spin_lock(&engine->i915->irq_lock);
engine->irq_disable(engine);
spin_unlock(&engine->i915->irq_lock);
+}
+
+void __intel_engine_disarm_breadcrumbs(struct intel_engine_cs *engine)
+{
+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
+
+ lockdep_assert_held(&b->irq_lock);
+ GEM_BUG_ON(b->irq_wait);
+
+ if (b->irq_enabled) {
+ irq_disable(engine);
+ b->irq_enabled = false;
+ }
- engine->breadcrumbs.irq_posted = false;
+ b->irq_armed = false;
}
-static void __intel_breadcrumbs_enable_irq(struct intel_breadcrumbs *b)
+void intel_engine_disarm_breadcrumbs(struct intel_engine_cs *engine)
{
- struct intel_engine_cs *engine =
- container_of(b, struct intel_engine_cs, breadcrumbs);
- struct drm_i915_private *i915 = engine->i915;
+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
+ struct intel_wait *wait, *n, *first;
- assert_spin_locked(&b->lock);
- if (b->rpm_wakelock)
+ if (!b->irq_armed)
return;
- /* Since we are waiting on a request, the GPU should be busy
- * and should have its own rpm reference. For completeness,
- * record an rpm reference for ourselves to cover the
- * interrupt we unmask.
+ /* We only disarm the irq when we are idle (all requests completed),
+ * so if the bottom-half remains asleep, it missed the request
+ * completion.
*/
- intel_runtime_pm_get_noresume(i915);
- b->rpm_wakelock = true;
- /* No interrupts? Kick the waiter every jiffie! */
- if (intel_irqs_enabled(i915)) {
- if (!test_bit(engine->id, &i915->gpu_error.test_irq_rings))
- irq_enable(engine);
- b->irq_enabled = true;
+ spin_lock_irq(&b->rb_lock);
+
+ spin_lock(&b->irq_lock);
+ first = fetch_and_zero(&b->irq_wait);
+ __intel_engine_disarm_breadcrumbs(engine);
+ spin_unlock(&b->irq_lock);
+
+ rbtree_postorder_for_each_entry_safe(wait, n, &b->waiters, node) {
+ RB_CLEAR_NODE(&wait->node);
+ if (wake_up_process(wait->tsk) && wait == first)
+ missed_breadcrumb(engine);
}
+ b->waiters = RB_ROOT;
- if (!b->irq_enabled ||
- test_bit(engine->id, &i915->gpu_error.missed_irq_rings)) {
+ spin_unlock_irq(&b->rb_lock);
+}
+
+static bool use_fake_irq(const struct intel_breadcrumbs *b)
+{
+ const struct intel_engine_cs *engine =
+ container_of(b, struct intel_engine_cs, breadcrumbs);
+
+ if (!test_bit(engine->id, &engine->i915->gpu_error.missed_irq_rings))
+ return false;
+
+ /* Only start with the heavy weight fake irq timer if we have not
+ * seen any interrupts since enabling it the first time. If the
+ * interrupts are still arriving, it means we made a mistake in our
+ * engine->seqno_barrier(), a timing error that should be transient
+ * and unlikely to reoccur.
+ */
+ return atomic_read(&engine->irq_count) == b->hangcheck_interrupts;
+}
+
+static void enable_fake_irq(struct intel_breadcrumbs *b)
+{
+ /* Ensure we never sleep indefinitely */
+ if (!b->irq_enabled || use_fake_irq(b))
mod_timer(&b->fake_irq, jiffies + 1);
- } else {
- /* Ensure we never sleep indefinitely */
- GEM_BUG_ON(!time_after(b->timeout, jiffies));
- mod_timer(&b->hangcheck, b->timeout);
- }
+ else
+ mod_timer(&b->hangcheck, wait_timeout());
}
-static void __intel_breadcrumbs_disable_irq(struct intel_breadcrumbs *b)
+static void __intel_breadcrumbs_enable_irq(struct intel_breadcrumbs *b)
{
struct intel_engine_cs *engine =
container_of(b, struct intel_engine_cs, breadcrumbs);
+ struct drm_i915_private *i915 = engine->i915;
- assert_spin_locked(&b->lock);
- if (!b->rpm_wakelock)
+ lockdep_assert_held(&b->irq_lock);
+ if (b->irq_armed)
return;
- if (b->irq_enabled) {
- irq_disable(engine);
- b->irq_enabled = false;
+ /* The breadcrumb irq will be disarmed on the interrupt after the
+ * waiters are signaled. This gives us a single interrupt window in
+ * which we can add a new waiter and avoid the cost of re-enabling
+ * the irq.
+ */
+ b->irq_armed = true;
+ GEM_BUG_ON(b->irq_enabled);
+
+ if (I915_SELFTEST_ONLY(b->mock)) {
+ /* For our mock objects we want to avoid interaction
+ * with the real hardware (which is not set up). So
+ * we simply pretend we have enabled the powerwell
+ * and the irq, and leave it up to the mock
+ * implementation to call intel_engine_wakeup()
+ * itself when it wants to simulate a user interrupt,
+ */
+ return;
}
- intel_runtime_pm_put(engine->i915);
- b->rpm_wakelock = false;
+ /* Since we are waiting on a request, the GPU should be busy
+ * and should have its own rpm reference. This is tracked
+ * by i915->gt.awake, we can forgo holding our own wakref
+ * for the interrupt as before i915->gt.awake is released (when
+ * the driver is idle) we disarm the breadcrumbs.
+ */
+
+ /* No interrupts? Kick the waiter every jiffie! */
+ if (intel_irqs_enabled(i915)) {
+ if (!test_bit(engine->id, &i915->gpu_error.test_irq_rings))
+ irq_enable(engine);
+ b->irq_enabled = true;
+ }
+
+ enable_fake_irq(b);
}
static inline struct intel_wait *to_wait(struct rb_node *node)
static inline void __intel_breadcrumbs_finish(struct intel_breadcrumbs *b,
struct intel_wait *wait)
{
- assert_spin_locked(&b->lock);
+ lockdep_assert_held(&b->rb_lock);
+ GEM_BUG_ON(b->irq_wait == wait);
/* This request is completed, so remove it from the tree, mark it as
- * complete, and *then* wake up the associated task.
+ * complete, and *then* wake up the associated task. N.B. when the
+ * task wakes up, it will find the empty rb_node, discern that it
+ * has already been removed from the tree and skip the serialisation
+ * of the b->rb_lock and b->irq_lock. This means that the destruction
+ * of the intel_wait is not serialised with the interrupt handler
+ * by the waiter - it must instead be serialised by the caller.
*/
rb_erase(&wait->node, &b->waiters);
RB_CLEAR_NODE(&wait->node);
wake_up_process(wait->tsk); /* implicit smp_wmb() */
}
+static inline void __intel_breadcrumbs_next(struct intel_engine_cs *engine,
+ struct rb_node *next)
+{
+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
+
+ spin_lock(&b->irq_lock);
+ GEM_BUG_ON(!b->irq_armed);
+ GEM_BUG_ON(!b->irq_wait);
+ b->irq_wait = to_wait(next);
+ spin_unlock(&b->irq_lock);
+
+ /* We always wake up the next waiter that takes over as the bottom-half
+ * as we may delegate not only the irq-seqno barrier to the next waiter
+ * but also the task of waking up concurrent waiters.
+ */
+ if (next)
+ wake_up_process(to_wait(next)->tsk);
+}
+
static bool __intel_engine_add_wait(struct intel_engine_cs *engine,
struct intel_wait *wait)
{
}
rb_link_node(&wait->node, parent, p);
rb_insert_color(&wait->node, &b->waiters);
- GEM_BUG_ON(!first && !rcu_access_pointer(b->irq_seqno_bh));
-
- if (completed) {
- struct rb_node *next = rb_next(completed);
-
- GEM_BUG_ON(!next && !first);
- if (next && next != &wait->node) {
- GEM_BUG_ON(first);
- b->timeout = wait_timeout();
- b->first_wait = to_wait(next);
- rcu_assign_pointer(b->irq_seqno_bh, b->first_wait->tsk);
- /* As there is a delay between reading the current
- * seqno, processing the completed tasks and selecting
- * the next waiter, we may have missed the interrupt
- * and so need for the next bottom-half to wakeup.
- *
- * Also as we enable the IRQ, we may miss the
- * interrupt for that seqno, so we have to wake up
- * the next bottom-half in order to do a coherent check
- * in case the seqno passed.
- */
- __intel_breadcrumbs_enable_irq(b);
- if (READ_ONCE(b->irq_posted))
- wake_up_process(to_wait(next)->tsk);
- }
-
- do {
- struct intel_wait *crumb = to_wait(completed);
- completed = rb_prev(completed);
- __intel_breadcrumbs_finish(b, crumb);
- } while (completed);
- }
if (first) {
- GEM_BUG_ON(rb_first(&b->waiters) != &wait->node);
- b->timeout = wait_timeout();
- b->first_wait = wait;
- rcu_assign_pointer(b->irq_seqno_bh, wait->tsk);
+ spin_lock(&b->irq_lock);
+ b->irq_wait = wait;
/* After assigning ourselves as the new bottom-half, we must
* perform a cursory check to prevent a missed interrupt.
* Either we miss the interrupt whilst programming the hardware,
* and so we miss the wake up.
*/
__intel_breadcrumbs_enable_irq(b);
+ spin_unlock(&b->irq_lock);
}
- GEM_BUG_ON(!rcu_access_pointer(b->irq_seqno_bh));
- GEM_BUG_ON(!b->first_wait);
- GEM_BUG_ON(rb_first(&b->waiters) != &b->first_wait->node);
+
+ if (completed) {
+ /* Advance the bottom-half (b->irq_wait) before we wake up
+ * the waiters who may scribble over their intel_wait
+ * just as the interrupt handler is dereferencing it via
+ * b->irq_wait.
+ */
+ if (!first) {
+ struct rb_node *next = rb_next(completed);
+ GEM_BUG_ON(next == &wait->node);
+ __intel_breadcrumbs_next(engine, next);
+ }
+
+ do {
+ struct intel_wait *crumb = to_wait(completed);
+ completed = rb_prev(completed);
+ __intel_breadcrumbs_finish(b, crumb);
+ } while (completed);
+ }
+
+ GEM_BUG_ON(!b->irq_wait);
+ GEM_BUG_ON(!b->irq_armed);
+ GEM_BUG_ON(rb_first(&b->waiters) != &b->irq_wait->node);
return first;
}
struct intel_breadcrumbs *b = &engine->breadcrumbs;
bool first;
- spin_lock_irq(&b->lock);
+ spin_lock_irq(&b->rb_lock);
first = __intel_engine_add_wait(engine, wait);
- spin_unlock_irq(&b->lock);
+ spin_unlock_irq(&b->rb_lock);
return first;
}
return tsk->prio;
}
-void intel_engine_remove_wait(struct intel_engine_cs *engine,
- struct intel_wait *wait)
+static void __intel_engine_remove_wait(struct intel_engine_cs *engine,
+ struct intel_wait *wait)
{
struct intel_breadcrumbs *b = &engine->breadcrumbs;
- /* Quick check to see if this waiter was already decoupled from
- * the tree by the bottom-half to avoid contention on the spinlock
- * by the herd.
- */
- if (RB_EMPTY_NODE(&wait->node))
- return;
-
- spin_lock_irq(&b->lock);
+ lockdep_assert_held(&b->rb_lock);
if (RB_EMPTY_NODE(&wait->node))
- goto out_unlock;
+ goto out;
- if (b->first_wait == wait) {
+ if (b->irq_wait == wait) {
const int priority = wakeup_priority(b, wait->tsk);
struct rb_node *next;
- GEM_BUG_ON(rcu_access_pointer(b->irq_seqno_bh) != wait->tsk);
-
/* We are the current bottom-half. Find the next candidate,
* the first waiter in the queue on the remaining oldest
* request. As multiple seqnos may complete in the time it
}
}
- if (next) {
- /* In our haste, we may have completed the first waiter
- * before we enabled the interrupt. Do so now as we
- * have a second waiter for a future seqno. Afterwards,
- * we have to wake up that waiter in case we missed
- * the interrupt, or if we have to handle an
- * exception rather than a seqno completion.
- */
- b->timeout = wait_timeout();
- b->first_wait = to_wait(next);
- rcu_assign_pointer(b->irq_seqno_bh, b->first_wait->tsk);
- if (b->first_wait->seqno != wait->seqno)
- __intel_breadcrumbs_enable_irq(b);
- wake_up_process(b->first_wait->tsk);
- } else {
- b->first_wait = NULL;
- rcu_assign_pointer(b->irq_seqno_bh, NULL);
- __intel_breadcrumbs_disable_irq(b);
- }
+ __intel_breadcrumbs_next(engine, next);
} else {
GEM_BUG_ON(rb_first(&b->waiters) == &wait->node);
}
GEM_BUG_ON(RB_EMPTY_NODE(&wait->node));
rb_erase(&wait->node, &b->waiters);
-out_unlock:
- GEM_BUG_ON(b->first_wait == wait);
+out:
+ GEM_BUG_ON(b->irq_wait == wait);
GEM_BUG_ON(rb_first(&b->waiters) !=
- (b->first_wait ? &b->first_wait->node : NULL));
- GEM_BUG_ON(!rcu_access_pointer(b->irq_seqno_bh) ^ RB_EMPTY_ROOT(&b->waiters));
- spin_unlock_irq(&b->lock);
+ (b->irq_wait ? &b->irq_wait->node : NULL));
+}
+
+void intel_engine_remove_wait(struct intel_engine_cs *engine,
+ struct intel_wait *wait)
+{
+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
+
+ /* Quick check to see if this waiter was already decoupled from
+ * the tree by the bottom-half to avoid contention on the spinlock
+ * by the herd.
+ */
+ if (RB_EMPTY_NODE(&wait->node)) {
+ GEM_BUG_ON(READ_ONCE(b->irq_wait) == wait);
+ return;
+ }
+
+ spin_lock_irq(&b->rb_lock);
+ __intel_engine_remove_wait(engine, wait);
+ spin_unlock_irq(&b->rb_lock);
+}
+
+static bool signal_valid(const struct drm_i915_gem_request *request)
+{
+ return intel_wait_check_request(&request->signaling.wait, request);
}
-static bool signal_complete(struct drm_i915_gem_request *request)
+static bool signal_complete(const struct drm_i915_gem_request *request)
{
if (!request)
return false;
* signalled that this wait is already completed.
*/
if (intel_wait_complete(&request->signaling.wait))
- return true;
+ return signal_valid(request);
/* Carefully check if the request is complete, giving time for the
* seqno to be visible or if the GPU hung.
* need to wait for a new interrupt from the GPU or for
* a new client.
*/
- request = READ_ONCE(b->first_signal);
+ rcu_read_lock();
+ request = rcu_dereference(b->first_signal);
+ if (request)
+ request = i915_gem_request_get_rcu(request);
+ rcu_read_unlock();
if (signal_complete(request)) {
- /* Wake up all other completed waiters and select the
- * next bottom-half for the next user interrupt.
- */
- intel_engine_remove_wait(engine,
- &request->signaling.wait);
-
local_bh_disable();
dma_fence_signal(&request->fence);
local_bh_enable(); /* kick start the tasklets */
+ spin_lock_irq(&b->rb_lock);
+
+ /* Wake up all other completed waiters and select the
+ * next bottom-half for the next user interrupt.
+ */
+ __intel_engine_remove_wait(engine,
+ &request->signaling.wait);
+
/* Find the next oldest signal. Note that as we have
* not been holding the lock, another client may
* have installed an even older signal than the one
* we just completed - so double check we are still
* the oldest before picking the next one.
*/
- spin_lock_irq(&b->lock);
- if (request == b->first_signal) {
+ if (request == rcu_access_pointer(b->first_signal)) {
struct rb_node *rb =
rb_next(&request->signaling.node);
- b->first_signal = rb ? to_signaler(rb) : NULL;
+ rcu_assign_pointer(b->first_signal,
+ rb ? to_signaler(rb) : NULL);
}
rb_erase(&request->signaling.node, &b->signals);
- spin_unlock_irq(&b->lock);
+ RB_CLEAR_NODE(&request->signaling.node);
+
+ spin_unlock_irq(&b->rb_lock);
i915_gem_request_put(request);
} else {
- if (kthread_should_stop())
+ DEFINE_WAIT(exec);
+
+ if (kthread_should_stop()) {
+ GEM_BUG_ON(request);
break;
+ }
+
+ if (request)
+ add_wait_queue(&request->execute, &exec);
schedule();
+
+ if (request)
+ remove_wait_queue(&request->execute, &exec);
+
+ if (kthread_should_park())
+ kthread_parkme();
}
+ i915_gem_request_put(request);
} while (1);
__set_current_state(TASK_RUNNING);
struct intel_breadcrumbs *b = &engine->breadcrumbs;
struct rb_node *parent, **p;
bool first, wakeup;
+ u32 seqno;
/* Note that we may be called from an interrupt handler on another
* device (e.g. nouveau signaling a fence completion causing us
* to submit a request, and so enable signaling). As such,
- * we need to make sure that all other users of b->lock protect
+ * we need to make sure that all other users of b->rb_lock protect
* against interrupts, i.e. use spin_lock_irqsave.
*/
/* locked by dma_fence_enable_sw_signaling() (irqsafe fence->lock) */
- assert_spin_locked(&request->lock);
- if (!request->global_seqno)
+ GEM_BUG_ON(!irqs_disabled());
+ lockdep_assert_held(&request->lock);
+
+ seqno = i915_gem_request_global_seqno(request);
+ if (!seqno)
return;
request->signaling.wait.tsk = b->signaler;
- request->signaling.wait.seqno = request->global_seqno;
+ request->signaling.wait.request = request;
+ request->signaling.wait.seqno = seqno;
i915_gem_request_get(request);
- spin_lock(&b->lock);
+ spin_lock(&b->rb_lock);
/* First add ourselves into the list of waiters, but register our
* bottom-half as the signaller thread. As per usual, only the oldest
p = &b->signals.rb_node;
while (*p) {
parent = *p;
- if (i915_seqno_passed(request->global_seqno,
- to_signaler(parent)->global_seqno)) {
+ if (i915_seqno_passed(seqno,
+ to_signaler(parent)->signaling.wait.seqno)) {
p = &parent->rb_right;
first = false;
} else {
rb_link_node(&request->signaling.node, parent, p);
rb_insert_color(&request->signaling.node, &b->signals);
if (first)
- smp_store_mb(b->first_signal, request);
+ rcu_assign_pointer(b->first_signal, request);
- spin_unlock(&b->lock);
+ spin_unlock(&b->rb_lock);
if (wakeup)
wake_up_process(b->signaler);
}
+void intel_engine_cancel_signaling(struct drm_i915_gem_request *request)
+{
+ struct intel_engine_cs *engine = request->engine;
+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
+
+ GEM_BUG_ON(!irqs_disabled());
+ lockdep_assert_held(&request->lock);
+ GEM_BUG_ON(!request->signaling.wait.seqno);
+
+ spin_lock(&b->rb_lock);
+
+ if (!RB_EMPTY_NODE(&request->signaling.node)) {
+ if (request == rcu_access_pointer(b->first_signal)) {
+ struct rb_node *rb =
+ rb_next(&request->signaling.node);
+ rcu_assign_pointer(b->first_signal,
+ rb ? to_signaler(rb) : NULL);
+ }
+ rb_erase(&request->signaling.node, &b->signals);
+ RB_CLEAR_NODE(&request->signaling.node);
+ i915_gem_request_put(request);
+ }
+
+ __intel_engine_remove_wait(engine, &request->signaling.wait);
+
+ spin_unlock(&b->rb_lock);
+
+ request->signaling.wait.seqno = 0;
+}
+
int intel_engine_init_breadcrumbs(struct intel_engine_cs *engine)
{
struct intel_breadcrumbs *b = &engine->breadcrumbs;
struct task_struct *tsk;
- spin_lock_init(&b->lock);
+ spin_lock_init(&b->rb_lock);
+ spin_lock_init(&b->irq_lock);
+
setup_timer(&b->fake_irq,
intel_breadcrumbs_fake_irq,
(unsigned long)engine);
struct intel_breadcrumbs *b = &engine->breadcrumbs;
cancel_fake_irq(engine);
- spin_lock_irq(&b->lock);
+ spin_lock_irq(&b->irq_lock);
- __intel_breadcrumbs_disable_irq(b);
- if (intel_engine_has_waiter(engine)) {
- b->timeout = wait_timeout();
- __intel_breadcrumbs_enable_irq(b);
- if (READ_ONCE(b->irq_posted))
- wake_up_process(b->first_wait->tsk);
- } else {
- /* sanitize the IMR and unmask any auxiliary interrupts */
+ if (b->irq_enabled)
+ irq_enable(engine);
+ else
irq_disable(engine);
- }
- spin_unlock_irq(&b->lock);
+ /* We set the IRQ_BREADCRUMB bit when we enable the irq presuming the
+ * GPU is active and may have already executed the MI_USER_INTERRUPT
+ * before the CPU is ready to receive. However, the engine is currently
+ * idle (we haven't started it yet), there is no possibility for a
+ * missed interrupt as we enabled the irq and so we can clear the
+ * immediate wakeup (until a real interrupt arrives for the waiter).
+ */
+ clear_bit(ENGINE_IRQ_BREADCRUMB, &engine->irq_posted);
+
+ if (b->irq_armed)
+ enable_fake_irq(b);
+
+ spin_unlock_irq(&b->irq_lock);
}
void intel_engine_fini_breadcrumbs(struct intel_engine_cs *engine)
struct intel_breadcrumbs *b = &engine->breadcrumbs;
/* The engines should be idle and all requests accounted for! */
- WARN_ON(READ_ONCE(b->first_wait));
+ WARN_ON(READ_ONCE(b->irq_wait));
WARN_ON(!RB_EMPTY_ROOT(&b->waiters));
- WARN_ON(READ_ONCE(b->first_signal));
+ WARN_ON(rcu_access_pointer(b->first_signal));
WARN_ON(!RB_EMPTY_ROOT(&b->signals));
if (!IS_ERR_OR_NULL(b->signaler))
cancel_fake_irq(engine);
}
-unsigned int intel_breadcrumbs_busy(struct drm_i915_private *i915)
+bool intel_breadcrumbs_busy(struct intel_engine_cs *engine)
{
- struct intel_engine_cs *engine;
- enum intel_engine_id id;
- unsigned int mask = 0;
-
- for_each_engine(engine, i915, id) {
- struct intel_breadcrumbs *b = &engine->breadcrumbs;
+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
+ bool busy = false;
- spin_lock_irq(&b->lock);
+ spin_lock_irq(&b->rb_lock);
- if (b->first_wait) {
- wake_up_process(b->first_wait->tsk);
- mask |= intel_engine_flag(engine);
- }
-
- if (b->first_signal) {
- wake_up_process(b->signaler);
- mask |= intel_engine_flag(engine);
- }
+ if (b->irq_wait) {
+ wake_up_process(b->irq_wait->tsk);
+ busy = true;
+ }
- spin_unlock_irq(&b->lock);
+ if (rcu_access_pointer(b->first_signal)) {
+ wake_up_process(b->signaler);
+ busy = true;
}
- return mask;
+ spin_unlock_irq(&b->rb_lock);
+
+ return busy;
}
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/intel_breadcrumbs.c"
+#endif
--- /dev/null
+/*
+ * Copyright © 2006-2017 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include "intel_drv.h"
+
+/**
+ * DOC: CDCLK / RAWCLK
+ *
+ * The display engine uses several different clocks to do its work. There
+ * are two main clocks involved that aren't directly related to the actual
+ * pixel clock or any symbol/bit clock of the actual output port. These
+ * are the core display clock (CDCLK) and RAWCLK.
+ *
+ * CDCLK clocks most of the display pipe logic, and thus its frequency
+ * must be high enough to support the rate at which pixels are flowing
+ * through the pipes. Downscaling must also be accounted as that increases
+ * the effective pixel rate.
+ *
+ * On several platforms the CDCLK frequency can be changed dynamically
+ * to minimize power consumption for a given display configuration.
+ * Typically changes to the CDCLK frequency require all the display pipes
+ * to be shut down while the frequency is being changed.
+ *
+ * On SKL+ the DMC will toggle the CDCLK off/on during DC5/6 entry/exit.
+ * DMC will not change the active CDCLK frequency however, so that part
+ * will still be performed by the driver directly.
+ *
+ * RAWCLK is a fixed frequency clock, often used by various auxiliary
+ * blocks such as AUX CH or backlight PWM. Hence the only thing we
+ * really need to know about RAWCLK is its frequency so that various
+ * dividers can be programmed correctly.
+ */
+
+static void fixed_133mhz_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ cdclk_state->cdclk = 133333;
+}
+
+static void fixed_200mhz_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ cdclk_state->cdclk = 200000;
+}
+
+static void fixed_266mhz_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ cdclk_state->cdclk = 266667;
+}
+
+static void fixed_333mhz_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ cdclk_state->cdclk = 333333;
+}
+
+static void fixed_400mhz_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ cdclk_state->cdclk = 400000;
+}
+
+static void fixed_450mhz_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ cdclk_state->cdclk = 450000;
+}
+
+static void i85x_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ struct pci_dev *pdev = dev_priv->drm.pdev;
+ u16 hpllcc = 0;
+
+ /*
+ * 852GM/852GMV only supports 133 MHz and the HPLLCC
+ * encoding is different :(
+ * FIXME is this the right way to detect 852GM/852GMV?
+ */
+ if (pdev->revision == 0x1) {
+ cdclk_state->cdclk = 133333;
+ return;
+ }
+
+ pci_bus_read_config_word(pdev->bus,
+ PCI_DEVFN(0, 3), HPLLCC, &hpllcc);
+
+ /* Assume that the hardware is in the high speed state. This
+ * should be the default.
+ */
+ switch (hpllcc & GC_CLOCK_CONTROL_MASK) {
+ case GC_CLOCK_133_200:
+ case GC_CLOCK_133_200_2:
+ case GC_CLOCK_100_200:
+ cdclk_state->cdclk = 200000;
+ break;
+ case GC_CLOCK_166_250:
+ cdclk_state->cdclk = 250000;
+ break;
+ case GC_CLOCK_100_133:
+ cdclk_state->cdclk = 133333;
+ break;
+ case GC_CLOCK_133_266:
+ case GC_CLOCK_133_266_2:
+ case GC_CLOCK_166_266:
+ cdclk_state->cdclk = 266667;
+ break;
+ }
+}
+
+static void i915gm_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ struct pci_dev *pdev = dev_priv->drm.pdev;
+ u16 gcfgc = 0;
+
+ pci_read_config_word(pdev, GCFGC, &gcfgc);
+
+ if (gcfgc & GC_LOW_FREQUENCY_ENABLE) {
+ cdclk_state->cdclk = 133333;
+ return;
+ }
+
+ switch (gcfgc & GC_DISPLAY_CLOCK_MASK) {
+ case GC_DISPLAY_CLOCK_333_320_MHZ:
+ cdclk_state->cdclk = 333333;
+ break;
+ default:
+ case GC_DISPLAY_CLOCK_190_200_MHZ:
+ cdclk_state->cdclk = 190000;
+ break;
+ }
+}
+
+static void i945gm_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ struct pci_dev *pdev = dev_priv->drm.pdev;
+ u16 gcfgc = 0;
+
+ pci_read_config_word(pdev, GCFGC, &gcfgc);
+
+ if (gcfgc & GC_LOW_FREQUENCY_ENABLE) {
+ cdclk_state->cdclk = 133333;
+ return;
+ }
+
+ switch (gcfgc & GC_DISPLAY_CLOCK_MASK) {
+ case GC_DISPLAY_CLOCK_333_320_MHZ:
+ cdclk_state->cdclk = 320000;
+ break;
+ default:
+ case GC_DISPLAY_CLOCK_190_200_MHZ:
+ cdclk_state->cdclk = 200000;
+ break;
+ }
+}
+
+static unsigned int intel_hpll_vco(struct drm_i915_private *dev_priv)
+{
+ static const unsigned int blb_vco[8] = {
+ [0] = 3200000,
+ [1] = 4000000,
+ [2] = 5333333,
+ [3] = 4800000,
+ [4] = 6400000,
+ };
+ static const unsigned int pnv_vco[8] = {
+ [0] = 3200000,
+ [1] = 4000000,
+ [2] = 5333333,
+ [3] = 4800000,
+ [4] = 2666667,
+ };
+ static const unsigned int cl_vco[8] = {
+ [0] = 3200000,
+ [1] = 4000000,
+ [2] = 5333333,
+ [3] = 6400000,
+ [4] = 3333333,
+ [5] = 3566667,
+ [6] = 4266667,
+ };
+ static const unsigned int elk_vco[8] = {
+ [0] = 3200000,
+ [1] = 4000000,
+ [2] = 5333333,
+ [3] = 4800000,
+ };
+ static const unsigned int ctg_vco[8] = {
+ [0] = 3200000,
+ [1] = 4000000,
+ [2] = 5333333,
+ [3] = 6400000,
+ [4] = 2666667,
+ [5] = 4266667,
+ };
+ const unsigned int *vco_table;
+ unsigned int vco;
+ uint8_t tmp = 0;
+
+ /* FIXME other chipsets? */
+ if (IS_GM45(dev_priv))
+ vco_table = ctg_vco;
+ else if (IS_G45(dev_priv))
+ vco_table = elk_vco;
+ else if (IS_I965GM(dev_priv))
+ vco_table = cl_vco;
+ else if (IS_PINEVIEW(dev_priv))
+ vco_table = pnv_vco;
+ else if (IS_G33(dev_priv))
+ vco_table = blb_vco;
+ else
+ return 0;
+
+ tmp = I915_READ(IS_MOBILE(dev_priv) ? HPLLVCO_MOBILE : HPLLVCO);
+
+ vco = vco_table[tmp & 0x7];
+ if (vco == 0)
+ DRM_ERROR("Bad HPLL VCO (HPLLVCO=0x%02x)\n", tmp);
+ else
+ DRM_DEBUG_KMS("HPLL VCO %u kHz\n", vco);
+
+ return vco;
+}
+
+static void g33_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ struct pci_dev *pdev = dev_priv->drm.pdev;
+ static const uint8_t div_3200[] = { 12, 10, 8, 7, 5, 16 };
+ static const uint8_t div_4000[] = { 14, 12, 10, 8, 6, 20 };
+ static const uint8_t div_4800[] = { 20, 14, 12, 10, 8, 24 };
+ static const uint8_t div_5333[] = { 20, 16, 12, 12, 8, 28 };
+ const uint8_t *div_table;
+ unsigned int cdclk_sel;
+ uint16_t tmp = 0;
+
+ cdclk_state->vco = intel_hpll_vco(dev_priv);
+
+ pci_read_config_word(pdev, GCFGC, &tmp);
+
+ cdclk_sel = (tmp >> 4) & 0x7;
+
+ if (cdclk_sel >= ARRAY_SIZE(div_3200))
+ goto fail;
+
+ switch (cdclk_state->vco) {
+ case 3200000:
+ div_table = div_3200;
+ break;
+ case 4000000:
+ div_table = div_4000;
+ break;
+ case 4800000:
+ div_table = div_4800;
+ break;
+ case 5333333:
+ div_table = div_5333;
+ break;
+ default:
+ goto fail;
+ }
+
+ cdclk_state->cdclk = DIV_ROUND_CLOSEST(cdclk_state->vco,
+ div_table[cdclk_sel]);
+ return;
+
+fail:
+ DRM_ERROR("Unable to determine CDCLK. HPLL VCO=%u kHz, CFGC=0x%08x\n",
+ cdclk_state->vco, tmp);
+ cdclk_state->cdclk = 190476;
+}
+
+static void pnv_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ struct pci_dev *pdev = dev_priv->drm.pdev;
+ u16 gcfgc = 0;
+
+ pci_read_config_word(pdev, GCFGC, &gcfgc);
+
+ switch (gcfgc & GC_DISPLAY_CLOCK_MASK) {
+ case GC_DISPLAY_CLOCK_267_MHZ_PNV:
+ cdclk_state->cdclk = 266667;
+ break;
+ case GC_DISPLAY_CLOCK_333_MHZ_PNV:
+ cdclk_state->cdclk = 333333;
+ break;
+ case GC_DISPLAY_CLOCK_444_MHZ_PNV:
+ cdclk_state->cdclk = 444444;
+ break;
+ case GC_DISPLAY_CLOCK_200_MHZ_PNV:
+ cdclk_state->cdclk = 200000;
+ break;
+ default:
+ DRM_ERROR("Unknown pnv display core clock 0x%04x\n", gcfgc);
+ case GC_DISPLAY_CLOCK_133_MHZ_PNV:
+ cdclk_state->cdclk = 133333;
+ break;
+ case GC_DISPLAY_CLOCK_167_MHZ_PNV:
+ cdclk_state->cdclk = 166667;
+ break;
+ }
+}
+
+static void i965gm_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ struct pci_dev *pdev = dev_priv->drm.pdev;
+ static const uint8_t div_3200[] = { 16, 10, 8 };
+ static const uint8_t div_4000[] = { 20, 12, 10 };
+ static const uint8_t div_5333[] = { 24, 16, 14 };
+ const uint8_t *div_table;
+ unsigned int cdclk_sel;
+ uint16_t tmp = 0;
+
+ cdclk_state->vco = intel_hpll_vco(dev_priv);
+
+ pci_read_config_word(pdev, GCFGC, &tmp);
+
+ cdclk_sel = ((tmp >> 8) & 0x1f) - 1;
+
+ if (cdclk_sel >= ARRAY_SIZE(div_3200))
+ goto fail;
+
+ switch (cdclk_state->vco) {
+ case 3200000:
+ div_table = div_3200;
+ break;
+ case 4000000:
+ div_table = div_4000;
+ break;
+ case 5333333:
+ div_table = div_5333;
+ break;
+ default:
+ goto fail;
+ }
+
+ cdclk_state->cdclk = DIV_ROUND_CLOSEST(cdclk_state->vco,
+ div_table[cdclk_sel]);
+ return;
+
+fail:
+ DRM_ERROR("Unable to determine CDCLK. HPLL VCO=%u kHz, CFGC=0x%04x\n",
+ cdclk_state->vco, tmp);
+ cdclk_state->cdclk = 200000;
+}
+
+static void gm45_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ struct pci_dev *pdev = dev_priv->drm.pdev;
+ unsigned int cdclk_sel;
+ uint16_t tmp = 0;
+
+ cdclk_state->vco = intel_hpll_vco(dev_priv);
+
+ pci_read_config_word(pdev, GCFGC, &tmp);
+
+ cdclk_sel = (tmp >> 12) & 0x1;
+
+ switch (cdclk_state->vco) {
+ case 2666667:
+ case 4000000:
+ case 5333333:
+ cdclk_state->cdclk = cdclk_sel ? 333333 : 222222;
+ break;
+ case 3200000:
+ cdclk_state->cdclk = cdclk_sel ? 320000 : 228571;
+ break;
+ default:
+ DRM_ERROR("Unable to determine CDCLK. HPLL VCO=%u, CFGC=0x%04x\n",
+ cdclk_state->vco, tmp);
+ cdclk_state->cdclk = 222222;
+ break;
+ }
+}
+
+static void hsw_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ uint32_t lcpll = I915_READ(LCPLL_CTL);
+ uint32_t freq = lcpll & LCPLL_CLK_FREQ_MASK;
+
+ if (lcpll & LCPLL_CD_SOURCE_FCLK)
+ cdclk_state->cdclk = 800000;
+ else if (I915_READ(FUSE_STRAP) & HSW_CDCLK_LIMIT)
+ cdclk_state->cdclk = 450000;
+ else if (freq == LCPLL_CLK_FREQ_450)
+ cdclk_state->cdclk = 450000;
+ else if (IS_HSW_ULT(dev_priv))
+ cdclk_state->cdclk = 337500;
+ else
+ cdclk_state->cdclk = 540000;
+}
+
+static int vlv_calc_cdclk(struct drm_i915_private *dev_priv,
+ int max_pixclk)
+{
+ int freq_320 = (dev_priv->hpll_freq << 1) % 320000 != 0 ?
+ 333333 : 320000;
+ int limit = IS_CHERRYVIEW(dev_priv) ? 95 : 90;
+
+ /*
+ * We seem to get an unstable or solid color picture at 200MHz.
+ * Not sure what's wrong. For now use 200MHz only when all pipes
+ * are off.
+ */
+ if (!IS_CHERRYVIEW(dev_priv) &&
+ max_pixclk > freq_320*limit/100)
+ return 400000;
+ else if (max_pixclk > 266667*limit/100)
+ return freq_320;
+ else if (max_pixclk > 0)
+ return 266667;
+ else
+ return 200000;
+}
+
+static void vlv_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ cdclk_state->vco = vlv_get_hpll_vco(dev_priv);
+ cdclk_state->cdclk = vlv_get_cck_clock(dev_priv, "cdclk",
+ CCK_DISPLAY_CLOCK_CONTROL,
+ cdclk_state->vco);
+}
+
+static void vlv_program_pfi_credits(struct drm_i915_private *dev_priv)
+{
+ unsigned int credits, default_credits;
+
+ if (IS_CHERRYVIEW(dev_priv))
+ default_credits = PFI_CREDIT(12);
+ else
+ default_credits = PFI_CREDIT(8);
+
+ if (dev_priv->cdclk.hw.cdclk >= dev_priv->czclk_freq) {
+ /* CHV suggested value is 31 or 63 */
+ if (IS_CHERRYVIEW(dev_priv))
+ credits = PFI_CREDIT_63;
+ else
+ credits = PFI_CREDIT(15);
+ } else {
+ credits = default_credits;
+ }
+
+ /*
+ * WA - write default credits before re-programming
+ * FIXME: should we also set the resend bit here?
+ */
+ I915_WRITE(GCI_CONTROL, VGA_FAST_MODE_DISABLE |
+ default_credits);
+
+ I915_WRITE(GCI_CONTROL, VGA_FAST_MODE_DISABLE |
+ credits | PFI_CREDIT_RESEND);
+
+ /*
+ * FIXME is this guaranteed to clear
+ * immediately or should we poll for it?
+ */
+ WARN_ON(I915_READ(GCI_CONTROL) & PFI_CREDIT_RESEND);
+}
+
+static void vlv_set_cdclk(struct drm_i915_private *dev_priv,
+ const struct intel_cdclk_state *cdclk_state)
+{
+ int cdclk = cdclk_state->cdclk;
+ u32 val, cmd;
+
+ if (cdclk >= 320000) /* jump to highest voltage for 400MHz too */
+ cmd = 2;
+ else if (cdclk == 266667)
+ cmd = 1;
+ else
+ cmd = 0;
+
+ mutex_lock(&dev_priv->rps.hw_lock);
+ val = vlv_punit_read(dev_priv, PUNIT_REG_DSPFREQ);
+ val &= ~DSPFREQGUAR_MASK;
+ val |= (cmd << DSPFREQGUAR_SHIFT);
+ vlv_punit_write(dev_priv, PUNIT_REG_DSPFREQ, val);
+ if (wait_for((vlv_punit_read(dev_priv, PUNIT_REG_DSPFREQ) &
+ DSPFREQSTAT_MASK) == (cmd << DSPFREQSTAT_SHIFT),
+ 50)) {
+ DRM_ERROR("timed out waiting for CDclk change\n");
+ }
+ mutex_unlock(&dev_priv->rps.hw_lock);
+
+ mutex_lock(&dev_priv->sb_lock);
+
+ if (cdclk == 400000) {
+ u32 divider;
+
+ divider = DIV_ROUND_CLOSEST(dev_priv->hpll_freq << 1,
+ cdclk) - 1;
+
+ /* adjust cdclk divider */
+ val = vlv_cck_read(dev_priv, CCK_DISPLAY_CLOCK_CONTROL);
+ val &= ~CCK_FREQUENCY_VALUES;
+ val |= divider;
+ vlv_cck_write(dev_priv, CCK_DISPLAY_CLOCK_CONTROL, val);
+
+ if (wait_for((vlv_cck_read(dev_priv, CCK_DISPLAY_CLOCK_CONTROL) &
+ CCK_FREQUENCY_STATUS) == (divider << CCK_FREQUENCY_STATUS_SHIFT),
+ 50))
+ DRM_ERROR("timed out waiting for CDclk change\n");
+ }
+
+ /* adjust self-refresh exit latency value */
+ val = vlv_bunit_read(dev_priv, BUNIT_REG_BISOC);
+ val &= ~0x7f;
+
+ /*
+ * For high bandwidth configs, we set a higher latency in the bunit
+ * so that the core display fetch happens in time to avoid underruns.
+ */
+ if (cdclk == 400000)
+ val |= 4500 / 250; /* 4.5 usec */
+ else
+ val |= 3000 / 250; /* 3.0 usec */
+ vlv_bunit_write(dev_priv, BUNIT_REG_BISOC, val);
+
+ mutex_unlock(&dev_priv->sb_lock);
+
+ intel_update_cdclk(dev_priv);
+
+ vlv_program_pfi_credits(dev_priv);
+}
+
+static void chv_set_cdclk(struct drm_i915_private *dev_priv,
+ const struct intel_cdclk_state *cdclk_state)
+{
+ int cdclk = cdclk_state->cdclk;
+ u32 val, cmd;
+
+ switch (cdclk) {
+ case 333333:
+ case 320000:
+ case 266667:
+ case 200000:
+ break;
+ default:
+ MISSING_CASE(cdclk);
+ return;
+ }
+
+ /*
+ * Specs are full of misinformation, but testing on actual
+ * hardware has shown that we just need to write the desired
+ * CCK divider into the Punit register.
+ */
+ cmd = DIV_ROUND_CLOSEST(dev_priv->hpll_freq << 1, cdclk) - 1;
+
+ mutex_lock(&dev_priv->rps.hw_lock);
+ val = vlv_punit_read(dev_priv, PUNIT_REG_DSPFREQ);
+ val &= ~DSPFREQGUAR_MASK_CHV;
+ val |= (cmd << DSPFREQGUAR_SHIFT_CHV);
+ vlv_punit_write(dev_priv, PUNIT_REG_DSPFREQ, val);
+ if (wait_for((vlv_punit_read(dev_priv, PUNIT_REG_DSPFREQ) &
+ DSPFREQSTAT_MASK_CHV) == (cmd << DSPFREQSTAT_SHIFT_CHV),
+ 50)) {
+ DRM_ERROR("timed out waiting for CDclk change\n");
+ }
+ mutex_unlock(&dev_priv->rps.hw_lock);
+
+ intel_update_cdclk(dev_priv);
+
+ vlv_program_pfi_credits(dev_priv);
+}
+
+static int bdw_calc_cdclk(int max_pixclk)
+{
+ if (max_pixclk > 540000)
+ return 675000;
+ else if (max_pixclk > 450000)
+ return 540000;
+ else if (max_pixclk > 337500)
+ return 450000;
+ else
+ return 337500;
+}
+
+static void bdw_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ uint32_t lcpll = I915_READ(LCPLL_CTL);
+ uint32_t freq = lcpll & LCPLL_CLK_FREQ_MASK;
+
+ if (lcpll & LCPLL_CD_SOURCE_FCLK)
+ cdclk_state->cdclk = 800000;
+ else if (I915_READ(FUSE_STRAP) & HSW_CDCLK_LIMIT)
+ cdclk_state->cdclk = 450000;
+ else if (freq == LCPLL_CLK_FREQ_450)
+ cdclk_state->cdclk = 450000;
+ else if (freq == LCPLL_CLK_FREQ_54O_BDW)
+ cdclk_state->cdclk = 540000;
+ else if (freq == LCPLL_CLK_FREQ_337_5_BDW)
+ cdclk_state->cdclk = 337500;
+ else
+ cdclk_state->cdclk = 675000;
+}
+
+static void bdw_set_cdclk(struct drm_i915_private *dev_priv,
+ const struct intel_cdclk_state *cdclk_state)
+{
+ int cdclk = cdclk_state->cdclk;
+ uint32_t val, data;
+ int ret;
+
+ if (WARN((I915_READ(LCPLL_CTL) &
+ (LCPLL_PLL_DISABLE | LCPLL_PLL_LOCK |
+ LCPLL_CD_CLOCK_DISABLE | LCPLL_ROOT_CD_CLOCK_DISABLE |
+ LCPLL_CD2X_CLOCK_DISABLE | LCPLL_POWER_DOWN_ALLOW |
+ LCPLL_CD_SOURCE_FCLK)) != LCPLL_PLL_LOCK,
+ "trying to change cdclk frequency with cdclk not enabled\n"))
+ return;
+
+ mutex_lock(&dev_priv->rps.hw_lock);
+ ret = sandybridge_pcode_write(dev_priv,
+ BDW_PCODE_DISPLAY_FREQ_CHANGE_REQ, 0x0);
+ mutex_unlock(&dev_priv->rps.hw_lock);
+ if (ret) {
+ DRM_ERROR("failed to inform pcode about cdclk change\n");
+ return;
+ }
+
+ val = I915_READ(LCPLL_CTL);
+ val |= LCPLL_CD_SOURCE_FCLK;
+ I915_WRITE(LCPLL_CTL, val);
+
+ if (wait_for_us(I915_READ(LCPLL_CTL) &
+ LCPLL_CD_SOURCE_FCLK_DONE, 1))
+ DRM_ERROR("Switching to FCLK failed\n");
+
+ val = I915_READ(LCPLL_CTL);
+ val &= ~LCPLL_CLK_FREQ_MASK;
+
+ switch (cdclk) {
+ case 450000:
+ val |= LCPLL_CLK_FREQ_450;
+ data = 0;
+ break;
+ case 540000:
+ val |= LCPLL_CLK_FREQ_54O_BDW;
+ data = 1;
+ break;
+ case 337500:
+ val |= LCPLL_CLK_FREQ_337_5_BDW;
+ data = 2;
+ break;
+ case 675000:
+ val |= LCPLL_CLK_FREQ_675_BDW;
+ data = 3;
+ break;
+ default:
+ WARN(1, "invalid cdclk frequency\n");
+ return;
+ }
+
+ I915_WRITE(LCPLL_CTL, val);
+
+ val = I915_READ(LCPLL_CTL);
+ val &= ~LCPLL_CD_SOURCE_FCLK;
+ I915_WRITE(LCPLL_CTL, val);
+
+ if (wait_for_us((I915_READ(LCPLL_CTL) &
+ LCPLL_CD_SOURCE_FCLK_DONE) == 0, 1))
+ DRM_ERROR("Switching back to LCPLL failed\n");
+
+ mutex_lock(&dev_priv->rps.hw_lock);
+ sandybridge_pcode_write(dev_priv, HSW_PCODE_DE_WRITE_FREQ_REQ, data);
+ mutex_unlock(&dev_priv->rps.hw_lock);
+
+ I915_WRITE(CDCLK_FREQ, DIV_ROUND_CLOSEST(cdclk, 1000) - 1);
+
+ intel_update_cdclk(dev_priv);
+
+ WARN(cdclk != dev_priv->cdclk.hw.cdclk,
+ "cdclk requested %d kHz but got %d kHz\n",
+ cdclk, dev_priv->cdclk.hw.cdclk);
+}
+
+static int skl_calc_cdclk(int max_pixclk, int vco)
+{
+ if (vco == 8640000) {
+ if (max_pixclk > 540000)
+ return 617143;
+ else if (max_pixclk > 432000)
+ return 540000;
+ else if (max_pixclk > 308571)
+ return 432000;
+ else
+ return 308571;
+ } else {
+ if (max_pixclk > 540000)
+ return 675000;
+ else if (max_pixclk > 450000)
+ return 540000;
+ else if (max_pixclk > 337500)
+ return 450000;
+ else
+ return 337500;
+ }
+}
+
+static void skl_dpll0_update(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ u32 val;
+
+ cdclk_state->ref = 24000;
+ cdclk_state->vco = 0;
+
+ val = I915_READ(LCPLL1_CTL);
+ if ((val & LCPLL_PLL_ENABLE) == 0)
+ return;
+
+ if (WARN_ON((val & LCPLL_PLL_LOCK) == 0))
+ return;
+
+ val = I915_READ(DPLL_CTRL1);
+
+ if (WARN_ON((val & (DPLL_CTRL1_HDMI_MODE(SKL_DPLL0) |
+ DPLL_CTRL1_SSC(SKL_DPLL0) |
+ DPLL_CTRL1_OVERRIDE(SKL_DPLL0))) !=
+ DPLL_CTRL1_OVERRIDE(SKL_DPLL0)))
+ return;
+
+ switch (val & DPLL_CTRL1_LINK_RATE_MASK(SKL_DPLL0)) {
+ case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_810, SKL_DPLL0):
+ case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1350, SKL_DPLL0):
+ case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1620, SKL_DPLL0):
+ case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_2700, SKL_DPLL0):
+ cdclk_state->vco = 8100000;
+ break;
+ case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1080, SKL_DPLL0):
+ case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_2160, SKL_DPLL0):
+ cdclk_state->vco = 8640000;
+ break;
+ default:
+ MISSING_CASE(val & DPLL_CTRL1_LINK_RATE_MASK(SKL_DPLL0));
+ break;
+ }
+}
+
+static void skl_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ u32 cdctl;
+
+ skl_dpll0_update(dev_priv, cdclk_state);
+
+ cdclk_state->cdclk = cdclk_state->ref;
+
+ if (cdclk_state->vco == 0)
+ return;
+
+ cdctl = I915_READ(CDCLK_CTL);
+
+ if (cdclk_state->vco == 8640000) {
+ switch (cdctl & CDCLK_FREQ_SEL_MASK) {
+ case CDCLK_FREQ_450_432:
+ cdclk_state->cdclk = 432000;
+ break;
+ case CDCLK_FREQ_337_308:
+ cdclk_state->cdclk = 308571;
+ break;
+ case CDCLK_FREQ_540:
+ cdclk_state->cdclk = 540000;
+ break;
+ case CDCLK_FREQ_675_617:
+ cdclk_state->cdclk = 617143;
+ break;
+ default:
+ MISSING_CASE(cdctl & CDCLK_FREQ_SEL_MASK);
+ break;
+ }
+ } else {
+ switch (cdctl & CDCLK_FREQ_SEL_MASK) {
+ case CDCLK_FREQ_450_432:
+ cdclk_state->cdclk = 450000;
+ break;
+ case CDCLK_FREQ_337_308:
+ cdclk_state->cdclk = 337500;
+ break;
+ case CDCLK_FREQ_540:
+ cdclk_state->cdclk = 540000;
+ break;
+ case CDCLK_FREQ_675_617:
+ cdclk_state->cdclk = 675000;
+ break;
+ default:
+ MISSING_CASE(cdctl & CDCLK_FREQ_SEL_MASK);
+ break;
+ }
+ }
+}
+
+/* convert from kHz to .1 fixpoint MHz with -1MHz offset */
+static int skl_cdclk_decimal(int cdclk)
+{
+ return DIV_ROUND_CLOSEST(cdclk - 1000, 500);
+}
+
+static void skl_set_preferred_cdclk_vco(struct drm_i915_private *dev_priv,
+ int vco)
+{
+ bool changed = dev_priv->skl_preferred_vco_freq != vco;
+
+ dev_priv->skl_preferred_vco_freq = vco;
+
+ if (changed)
+ intel_update_max_cdclk(dev_priv);
+}
+
+static void skl_dpll0_enable(struct drm_i915_private *dev_priv, int vco)
+{
+ int min_cdclk = skl_calc_cdclk(0, vco);
+ u32 val;
+
+ WARN_ON(vco != 8100000 && vco != 8640000);
+
+ /* select the minimum CDCLK before enabling DPLL 0 */
+ val = CDCLK_FREQ_337_308 | skl_cdclk_decimal(min_cdclk);
+ I915_WRITE(CDCLK_CTL, val);
+ POSTING_READ(CDCLK_CTL);
+
+ /*
+ * We always enable DPLL0 with the lowest link rate possible, but still
+ * taking into account the VCO required to operate the eDP panel at the
+ * desired frequency. The usual DP link rates operate with a VCO of
+ * 8100 while the eDP 1.4 alternate link rates need a VCO of 8640.
+ * The modeset code is responsible for the selection of the exact link
+ * rate later on, with the constraint of choosing a frequency that
+ * works with vco.
+ */
+ val = I915_READ(DPLL_CTRL1);
+
+ val &= ~(DPLL_CTRL1_HDMI_MODE(SKL_DPLL0) | DPLL_CTRL1_SSC(SKL_DPLL0) |
+ DPLL_CTRL1_LINK_RATE_MASK(SKL_DPLL0));
+ val |= DPLL_CTRL1_OVERRIDE(SKL_DPLL0);
+ if (vco == 8640000)
+ val |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1080,
+ SKL_DPLL0);
+ else
+ val |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_810,
+ SKL_DPLL0);
+
+ I915_WRITE(DPLL_CTRL1, val);
+ POSTING_READ(DPLL_CTRL1);
+
+ I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) | LCPLL_PLL_ENABLE);
+
+ if (intel_wait_for_register(dev_priv,
+ LCPLL1_CTL, LCPLL_PLL_LOCK, LCPLL_PLL_LOCK,
+ 5))
+ DRM_ERROR("DPLL0 not locked\n");
+
+ dev_priv->cdclk.hw.vco = vco;
+
+ /* We'll want to keep using the current vco from now on. */
+ skl_set_preferred_cdclk_vco(dev_priv, vco);
+}
+
+static void skl_dpll0_disable(struct drm_i915_private *dev_priv)
+{
+ I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) & ~LCPLL_PLL_ENABLE);
+ if (intel_wait_for_register(dev_priv,
+ LCPLL1_CTL, LCPLL_PLL_LOCK, 0,
+ 1))
+ DRM_ERROR("Couldn't disable DPLL0\n");
+
+ dev_priv->cdclk.hw.vco = 0;
+}
+
+static void skl_set_cdclk(struct drm_i915_private *dev_priv,
+ const struct intel_cdclk_state *cdclk_state)
+{
+ int cdclk = cdclk_state->cdclk;
+ int vco = cdclk_state->vco;
+ u32 freq_select, pcu_ack;
+ int ret;
+
+ WARN_ON((cdclk == 24000) != (vco == 0));
+
+ mutex_lock(&dev_priv->rps.hw_lock);
+ ret = skl_pcode_request(dev_priv, SKL_PCODE_CDCLK_CONTROL,
+ SKL_CDCLK_PREPARE_FOR_CHANGE,
+ SKL_CDCLK_READY_FOR_CHANGE,
+ SKL_CDCLK_READY_FOR_CHANGE, 3);
+ mutex_unlock(&dev_priv->rps.hw_lock);
+ if (ret) {
+ DRM_ERROR("Failed to inform PCU about cdclk change (%d)\n",
+ ret);
+ return;
+ }
+
+ /* set CDCLK_CTL */
+ switch (cdclk) {
+ case 450000:
+ case 432000:
+ freq_select = CDCLK_FREQ_450_432;
+ pcu_ack = 1;
+ break;
+ case 540000:
+ freq_select = CDCLK_FREQ_540;
+ pcu_ack = 2;
+ break;
+ case 308571:
+ case 337500:
+ default:
+ freq_select = CDCLK_FREQ_337_308;
+ pcu_ack = 0;
+ break;
+ case 617143:
+ case 675000:
+ freq_select = CDCLK_FREQ_675_617;
+ pcu_ack = 3;
+ break;
+ }
+
+ if (dev_priv->cdclk.hw.vco != 0 &&
+ dev_priv->cdclk.hw.vco != vco)
+ skl_dpll0_disable(dev_priv);
+
+ if (dev_priv->cdclk.hw.vco != vco)
+ skl_dpll0_enable(dev_priv, vco);
+
+ I915_WRITE(CDCLK_CTL, freq_select | skl_cdclk_decimal(cdclk));
+ POSTING_READ(CDCLK_CTL);
+
+ /* inform PCU of the change */
+ mutex_lock(&dev_priv->rps.hw_lock);
+ sandybridge_pcode_write(dev_priv, SKL_PCODE_CDCLK_CONTROL, pcu_ack);
+ mutex_unlock(&dev_priv->rps.hw_lock);
+
+ intel_update_cdclk(dev_priv);
+}
+
+static void skl_sanitize_cdclk(struct drm_i915_private *dev_priv)
+{
+ uint32_t cdctl, expected;
+
+ /*
+ * check if the pre-os initialized the display
+ * There is SWF18 scratchpad register defined which is set by the
+ * pre-os which can be used by the OS drivers to check the status
+ */
+ if ((I915_READ(SWF_ILK(0x18)) & 0x00FFFFFF) == 0)
+ goto sanitize;
+
+ intel_update_cdclk(dev_priv);
+ /* Is PLL enabled and locked ? */
+ if (dev_priv->cdclk.hw.vco == 0 ||
+ dev_priv->cdclk.hw.cdclk == dev_priv->cdclk.hw.ref)
+ goto sanitize;
+
+ /* DPLL okay; verify the cdclock
+ *
+ * Noticed in some instances that the freq selection is correct but
+ * decimal part is programmed wrong from BIOS where pre-os does not
+ * enable display. Verify the same as well.
+ */
+ cdctl = I915_READ(CDCLK_CTL);
+ expected = (cdctl & CDCLK_FREQ_SEL_MASK) |
+ skl_cdclk_decimal(dev_priv->cdclk.hw.cdclk);
+ if (cdctl == expected)
+ /* All well; nothing to sanitize */
+ return;
+
+sanitize:
+ DRM_DEBUG_KMS("Sanitizing cdclk programmed by pre-os\n");
+
+ /* force cdclk programming */
+ dev_priv->cdclk.hw.cdclk = 0;
+ /* force full PLL disable + enable */
+ dev_priv->cdclk.hw.vco = -1;
+}
+
+/**
+ * skl_init_cdclk - Initialize CDCLK on SKL
+ * @dev_priv: i915 device
+ *
+ * Initialize CDCLK for SKL and derivatives. This is generally
+ * done only during the display core initialization sequence,
+ * after which the DMC will take care of turning CDCLK off/on
+ * as needed.
+ */
+void skl_init_cdclk(struct drm_i915_private *dev_priv)
+{
+ struct intel_cdclk_state cdclk_state;
+
+ skl_sanitize_cdclk(dev_priv);
+
+ if (dev_priv->cdclk.hw.cdclk != 0 &&
+ dev_priv->cdclk.hw.vco != 0) {
+ /*
+ * Use the current vco as our initial
+ * guess as to what the preferred vco is.
+ */
+ if (dev_priv->skl_preferred_vco_freq == 0)
+ skl_set_preferred_cdclk_vco(dev_priv,
+ dev_priv->cdclk.hw.vco);
+ return;
+ }
+
+ cdclk_state = dev_priv->cdclk.hw;
+
+ cdclk_state.vco = dev_priv->skl_preferred_vco_freq;
+ if (cdclk_state.vco == 0)
+ cdclk_state.vco = 8100000;
+ cdclk_state.cdclk = skl_calc_cdclk(0, cdclk_state.vco);
+
+ skl_set_cdclk(dev_priv, &cdclk_state);
+}
+
+/**
+ * skl_uninit_cdclk - Uninitialize CDCLK on SKL
+ * @dev_priv: i915 device
+ *
+ * Uninitialize CDCLK for SKL and derivatives. This is done only
+ * during the display core uninitialization sequence.
+ */
+void skl_uninit_cdclk(struct drm_i915_private *dev_priv)
+{
+ struct intel_cdclk_state cdclk_state = dev_priv->cdclk.hw;
+
+ cdclk_state.cdclk = cdclk_state.ref;
+ cdclk_state.vco = 0;
+
+ skl_set_cdclk(dev_priv, &cdclk_state);
+}
+
+static int bxt_calc_cdclk(int max_pixclk)
+{
+ if (max_pixclk > 576000)
+ return 624000;
+ else if (max_pixclk > 384000)
+ return 576000;
+ else if (max_pixclk > 288000)
+ return 384000;
+ else if (max_pixclk > 144000)
+ return 288000;
+ else
+ return 144000;
+}
+
+static int glk_calc_cdclk(int max_pixclk)
+{
+ if (max_pixclk > 2 * 158400)
+ return 316800;
+ else if (max_pixclk > 2 * 79200)
+ return 158400;
+ else
+ return 79200;
+}
+
+static int bxt_de_pll_vco(struct drm_i915_private *dev_priv, int cdclk)
+{
+ int ratio;
+
+ if (cdclk == dev_priv->cdclk.hw.ref)
+ return 0;
+
+ switch (cdclk) {
+ default:
+ MISSING_CASE(cdclk);
+ case 144000:
+ case 288000:
+ case 384000:
+ case 576000:
+ ratio = 60;
+ break;
+ case 624000:
+ ratio = 65;
+ break;
+ }
+
+ return dev_priv->cdclk.hw.ref * ratio;
+}
+
+static int glk_de_pll_vco(struct drm_i915_private *dev_priv, int cdclk)
+{
+ int ratio;
+
+ if (cdclk == dev_priv->cdclk.hw.ref)
+ return 0;
+
+ switch (cdclk) {
+ default:
+ MISSING_CASE(cdclk);
+ case 79200:
+ case 158400:
+ case 316800:
+ ratio = 33;
+ break;
+ }
+
+ return dev_priv->cdclk.hw.ref * ratio;
+}
+
+static void bxt_de_pll_update(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ u32 val;
+
+ cdclk_state->ref = 19200;
+ cdclk_state->vco = 0;
+
+ val = I915_READ(BXT_DE_PLL_ENABLE);
+ if ((val & BXT_DE_PLL_PLL_ENABLE) == 0)
+ return;
+
+ if (WARN_ON((val & BXT_DE_PLL_LOCK) == 0))
+ return;
+
+ val = I915_READ(BXT_DE_PLL_CTL);
+ cdclk_state->vco = (val & BXT_DE_PLL_RATIO_MASK) * cdclk_state->ref;
+}
+
+static void bxt_get_cdclk(struct drm_i915_private *dev_priv,
+ struct intel_cdclk_state *cdclk_state)
+{
+ u32 divider;
+ int div;
+
+ bxt_de_pll_update(dev_priv, cdclk_state);
+
+ cdclk_state->cdclk = cdclk_state->ref;
+
+ if (cdclk_state->vco == 0)
+ return;
+
+ divider = I915_READ(CDCLK_CTL) & BXT_CDCLK_CD2X_DIV_SEL_MASK;
+
+ switch (divider) {
+ case BXT_CDCLK_CD2X_DIV_SEL_1:
+ div = 2;
+ break;
+ case BXT_CDCLK_CD2X_DIV_SEL_1_5:
+ WARN(IS_GEMINILAKE(dev_priv), "Unsupported divider\n");
+ div = 3;
+ break;
+ case BXT_CDCLK_CD2X_DIV_SEL_2:
+ div = 4;
+ break;
+ case BXT_CDCLK_CD2X_DIV_SEL_4:
+ div = 8;
+ break;
+ default:
+ MISSING_CASE(divider);
+ return;
+ }
+
+ cdclk_state->cdclk = DIV_ROUND_CLOSEST(cdclk_state->vco, div);
+}
+
+static void bxt_de_pll_disable(struct drm_i915_private *dev_priv)
+{
+ I915_WRITE(BXT_DE_PLL_ENABLE, 0);
+
+ /* Timeout 200us */
+ if (intel_wait_for_register(dev_priv,
+ BXT_DE_PLL_ENABLE, BXT_DE_PLL_LOCK, 0,
+ 1))
+ DRM_ERROR("timeout waiting for DE PLL unlock\n");
+
+ dev_priv->cdclk.hw.vco = 0;
+}
+
+static void bxt_de_pll_enable(struct drm_i915_private *dev_priv, int vco)
+{
+ int ratio = DIV_ROUND_CLOSEST(vco, dev_priv->cdclk.hw.ref);
+ u32 val;
+
+ val = I915_READ(BXT_DE_PLL_CTL);
+ val &= ~BXT_DE_PLL_RATIO_MASK;
+ val |= BXT_DE_PLL_RATIO(ratio);
+ I915_WRITE(BXT_DE_PLL_CTL, val);
+
+ I915_WRITE(BXT_DE_PLL_ENABLE, BXT_DE_PLL_PLL_ENABLE);
+
+ /* Timeout 200us */
+ if (intel_wait_for_register(dev_priv,
+ BXT_DE_PLL_ENABLE,
+ BXT_DE_PLL_LOCK,
+ BXT_DE_PLL_LOCK,
+ 1))
+ DRM_ERROR("timeout waiting for DE PLL lock\n");
+
+ dev_priv->cdclk.hw.vco = vco;
+}
+
+static void bxt_set_cdclk(struct drm_i915_private *dev_priv,
+ const struct intel_cdclk_state *cdclk_state)
+{
+ int cdclk = cdclk_state->cdclk;
+ int vco = cdclk_state->vco;
+ u32 val, divider;
+ int ret;
+
+ /* cdclk = vco / 2 / div{1,1.5,2,4} */
+ switch (DIV_ROUND_CLOSEST(vco, cdclk)) {
+ case 8:
+ divider = BXT_CDCLK_CD2X_DIV_SEL_4;
+ break;
+ case 4:
+ divider = BXT_CDCLK_CD2X_DIV_SEL_2;
+ break;
+ case 3:
+ WARN(IS_GEMINILAKE(dev_priv), "Unsupported divider\n");
+ divider = BXT_CDCLK_CD2X_DIV_SEL_1_5;
+ break;
+ case 2:
+ divider = BXT_CDCLK_CD2X_DIV_SEL_1;
+ break;
+ default:
+ WARN_ON(cdclk != dev_priv->cdclk.hw.ref);
+ WARN_ON(vco != 0);
+
+ divider = BXT_CDCLK_CD2X_DIV_SEL_1;
+ break;
+ }
+
+ /* Inform power controller of upcoming frequency change */
+ mutex_lock(&dev_priv->rps.hw_lock);
+ ret = sandybridge_pcode_write(dev_priv, HSW_PCODE_DE_WRITE_FREQ_REQ,
+ 0x80000000);
+ mutex_unlock(&dev_priv->rps.hw_lock);
+
+ if (ret) {
+ DRM_ERROR("PCode CDCLK freq change notify failed (err %d, freq %d)\n",
+ ret, cdclk);
+ return;
+ }
+
+ if (dev_priv->cdclk.hw.vco != 0 &&
+ dev_priv->cdclk.hw.vco != vco)
+ bxt_de_pll_disable(dev_priv);
+
+ if (dev_priv->cdclk.hw.vco != vco)
+ bxt_de_pll_enable(dev_priv, vco);
+
+ val = divider | skl_cdclk_decimal(cdclk);
+ /*
+ * FIXME if only the cd2x divider needs changing, it could be done
+ * without shutting off the pipe (if only one pipe is active).
+ */
+ val |= BXT_CDCLK_CD2X_PIPE_NONE;
+ /*
+ * Disable SSA Precharge when CD clock frequency < 500 MHz,
+ * enable otherwise.
+ */
+ if (cdclk >= 500000)
+ val |= BXT_CDCLK_SSA_PRECHARGE_ENABLE;
+ I915_WRITE(CDCLK_CTL, val);
+
+ mutex_lock(&dev_priv->rps.hw_lock);
+ ret = sandybridge_pcode_write(dev_priv, HSW_PCODE_DE_WRITE_FREQ_REQ,
+ DIV_ROUND_UP(cdclk, 25000));
+ mutex_unlock(&dev_priv->rps.hw_lock);
+
+ if (ret) {
+ DRM_ERROR("PCode CDCLK freq set failed, (err %d, freq %d)\n",
+ ret, cdclk);
+ return;
+ }
+
+ intel_update_cdclk(dev_priv);
+}
+
+static void bxt_sanitize_cdclk(struct drm_i915_private *dev_priv)
+{
+ u32 cdctl, expected;
+
+ intel_update_cdclk(dev_priv);
+
+ if (dev_priv->cdclk.hw.vco == 0 ||
+ dev_priv->cdclk.hw.cdclk == dev_priv->cdclk.hw.ref)
+ goto sanitize;
+
+ /* DPLL okay; verify the cdclock
+ *
+ * Some BIOS versions leave an incorrect decimal frequency value and
+ * set reserved MBZ bits in CDCLK_CTL at least during exiting from S4,
+ * so sanitize this register.
+ */
+ cdctl = I915_READ(CDCLK_CTL);
+ /*
+ * Let's ignore the pipe field, since BIOS could have configured the
+ * dividers both synching to an active pipe, or asynchronously
+ * (PIPE_NONE).
+ */
+ cdctl &= ~BXT_CDCLK_CD2X_PIPE_NONE;
+
+ expected = (cdctl & BXT_CDCLK_CD2X_DIV_SEL_MASK) |
+ skl_cdclk_decimal(dev_priv->cdclk.hw.cdclk);
+ /*
+ * Disable SSA Precharge when CD clock frequency < 500 MHz,
+ * enable otherwise.
+ */
+ if (dev_priv->cdclk.hw.cdclk >= 500000)
+ expected |= BXT_CDCLK_SSA_PRECHARGE_ENABLE;
+
+ if (cdctl == expected)
+ /* All well; nothing to sanitize */
+ return;
+
+sanitize:
+ DRM_DEBUG_KMS("Sanitizing cdclk programmed by pre-os\n");
+
+ /* force cdclk programming */
+ dev_priv->cdclk.hw.cdclk = 0;
+
+ /* force full PLL disable + enable */
+ dev_priv->cdclk.hw.vco = -1;
+}
+
+/**
+ * bxt_init_cdclk - Initialize CDCLK on BXT
+ * @dev_priv: i915 device
+ *
+ * Initialize CDCLK for BXT and derivatives. This is generally
+ * done only during the display core initialization sequence,
+ * after which the DMC will take care of turning CDCLK off/on
+ * as needed.
+ */
+void bxt_init_cdclk(struct drm_i915_private *dev_priv)
+{
+ struct intel_cdclk_state cdclk_state;
+
+ bxt_sanitize_cdclk(dev_priv);
+
+ if (dev_priv->cdclk.hw.cdclk != 0 &&
+ dev_priv->cdclk.hw.vco != 0)
+ return;
+
+ cdclk_state = dev_priv->cdclk.hw;
+
+ /*
+ * FIXME:
+ * - The initial CDCLK needs to be read from VBT.
+ * Need to make this change after VBT has changes for BXT.
+ */
+ if (IS_GEMINILAKE(dev_priv)) {
+ cdclk_state.cdclk = glk_calc_cdclk(0);
+ cdclk_state.vco = glk_de_pll_vco(dev_priv, cdclk_state.cdclk);
+ } else {
+ cdclk_state.cdclk = bxt_calc_cdclk(0);
+ cdclk_state.vco = bxt_de_pll_vco(dev_priv, cdclk_state.cdclk);
+ }
+
+ bxt_set_cdclk(dev_priv, &cdclk_state);
+}
+
+/**
+ * bxt_uninit_cdclk - Uninitialize CDCLK on BXT
+ * @dev_priv: i915 device
+ *
+ * Uninitialize CDCLK for BXT and derivatives. This is done only
+ * during the display core uninitialization sequence.
+ */
+void bxt_uninit_cdclk(struct drm_i915_private *dev_priv)
+{
+ struct intel_cdclk_state cdclk_state = dev_priv->cdclk.hw;
+
+ cdclk_state.cdclk = cdclk_state.ref;
+ cdclk_state.vco = 0;
+
+ bxt_set_cdclk(dev_priv, &cdclk_state);
+}
+
+/**
+ * intel_cdclk_state_compare - Determine if two CDCLK states differ
+ * @a: first CDCLK state
+ * @b: second CDCLK state
+ *
+ * Returns:
+ * True if the CDCLK states are identical, false if they differ.
+ */
+bool intel_cdclk_state_compare(const struct intel_cdclk_state *a,
+ const struct intel_cdclk_state *b)
+{
+ return memcmp(a, b, sizeof(*a)) == 0;
+}
+
+/**
+ * intel_set_cdclk - Push the CDCLK state to the hardware
+ * @dev_priv: i915 device
+ * @cdclk_state: new CDCLK state
+ *
+ * Program the hardware based on the passed in CDCLK state,
+ * if necessary.
+ */
+void intel_set_cdclk(struct drm_i915_private *dev_priv,
+ const struct intel_cdclk_state *cdclk_state)
+{
+ if (intel_cdclk_state_compare(&dev_priv->cdclk.hw, cdclk_state))
+ return;
+
+ if (WARN_ON_ONCE(!dev_priv->display.set_cdclk))
+ return;
+
+ DRM_DEBUG_DRIVER("Changing CDCLK to %d kHz, VCO %d kHz, ref %d kHz\n",
+ cdclk_state->cdclk, cdclk_state->vco,
+ cdclk_state->ref);
+
+ dev_priv->display.set_cdclk(dev_priv, cdclk_state);
+}
+
+static int bdw_adjust_min_pipe_pixel_rate(struct intel_crtc_state *crtc_state,
+ int pixel_rate)
+{
+ struct drm_i915_private *dev_priv =
+ to_i915(crtc_state->base.crtc->dev);
+
+ /* pixel rate mustn't exceed 95% of cdclk with IPS on BDW */
+ if (IS_BROADWELL(dev_priv) && crtc_state->ips_enabled)
+ pixel_rate = DIV_ROUND_UP(pixel_rate * 100, 95);
+
+ /* BSpec says "Do not use DisplayPort with CDCLK less than
+ * 432 MHz, audio enabled, port width x4, and link rate
+ * HBR2 (5.4 GHz), or else there may be audio corruption or
+ * screen corruption."
+ */
+ if (intel_crtc_has_dp_encoder(crtc_state) &&
+ crtc_state->has_audio &&
+ crtc_state->port_clock >= 540000 &&
+ crtc_state->lane_count == 4)
+ pixel_rate = max(432000, pixel_rate);
+
+ return pixel_rate;
+}
+
+/* compute the max rate for new configuration */
+static int intel_max_pixel_rate(struct drm_atomic_state *state)
+{
+ struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
+ struct drm_i915_private *dev_priv = to_i915(state->dev);
+ struct drm_crtc *crtc;
+ struct drm_crtc_state *cstate;
+ struct intel_crtc_state *crtc_state;
+ unsigned int max_pixel_rate = 0, i;
+ enum pipe pipe;
+
+ memcpy(intel_state->min_pixclk, dev_priv->min_pixclk,
+ sizeof(intel_state->min_pixclk));
+
+ for_each_new_crtc_in_state(state, crtc, cstate, i) {
+ int pixel_rate;
+
+ crtc_state = to_intel_crtc_state(cstate);
+ if (!crtc_state->base.enable) {
+ intel_state->min_pixclk[i] = 0;
+ continue;
+ }
+
+ pixel_rate = crtc_state->pixel_rate;
+
+ if (IS_BROADWELL(dev_priv) || IS_GEN9(dev_priv))
+ pixel_rate =
+ bdw_adjust_min_pipe_pixel_rate(crtc_state,
+ pixel_rate);
+
+ intel_state->min_pixclk[i] = pixel_rate;
+ }
+
+ for_each_pipe(dev_priv, pipe)
+ max_pixel_rate = max(intel_state->min_pixclk[pipe],
+ max_pixel_rate);
+
+ return max_pixel_rate;
+}
+
+static int vlv_modeset_calc_cdclk(struct drm_atomic_state *state)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->dev);
+ int max_pixclk = intel_max_pixel_rate(state);
+ struct intel_atomic_state *intel_state =
+ to_intel_atomic_state(state);
+ int cdclk;
+
+ cdclk = vlv_calc_cdclk(dev_priv, max_pixclk);
+
+ if (cdclk > dev_priv->max_cdclk_freq) {
+ DRM_DEBUG_KMS("requested cdclk (%d kHz) exceeds max (%d kHz)\n",
+ cdclk, dev_priv->max_cdclk_freq);
+ return -EINVAL;
+ }
+
+ intel_state->cdclk.logical.cdclk = cdclk;
+
+ if (!intel_state->active_crtcs) {
+ cdclk = vlv_calc_cdclk(dev_priv, 0);
+
+ intel_state->cdclk.actual.cdclk = cdclk;
+ } else {
+ intel_state->cdclk.actual =
+ intel_state->cdclk.logical;
+ }
+
+ return 0;
+}
+
+static int bdw_modeset_calc_cdclk(struct drm_atomic_state *state)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->dev);
+ struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
+ int max_pixclk = intel_max_pixel_rate(state);
+ int cdclk;
+
+ /*
+ * FIXME should also account for plane ratio
+ * once 64bpp pixel formats are supported.
+ */
+ cdclk = bdw_calc_cdclk(max_pixclk);
+
+ if (cdclk > dev_priv->max_cdclk_freq) {
+ DRM_DEBUG_KMS("requested cdclk (%d kHz) exceeds max (%d kHz)\n",
+ cdclk, dev_priv->max_cdclk_freq);
+ return -EINVAL;
+ }
+
+ intel_state->cdclk.logical.cdclk = cdclk;
+
+ if (!intel_state->active_crtcs) {
+ cdclk = bdw_calc_cdclk(0);
+
+ intel_state->cdclk.actual.cdclk = cdclk;
+ } else {
+ intel_state->cdclk.actual =
+ intel_state->cdclk.logical;
+ }
+
+ return 0;
+}
+
+static int skl_modeset_calc_cdclk(struct drm_atomic_state *state)
+{
+ struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
+ struct drm_i915_private *dev_priv = to_i915(state->dev);
+ const int max_pixclk = intel_max_pixel_rate(state);
+ int cdclk, vco;
+
+ vco = intel_state->cdclk.logical.vco;
+ if (!vco)
+ vco = dev_priv->skl_preferred_vco_freq;
+
+ /*
+ * FIXME should also account for plane ratio
+ * once 64bpp pixel formats are supported.
+ */
+ cdclk = skl_calc_cdclk(max_pixclk, vco);
+
+ if (cdclk > dev_priv->max_cdclk_freq) {
+ DRM_DEBUG_KMS("requested cdclk (%d kHz) exceeds max (%d kHz)\n",
+ cdclk, dev_priv->max_cdclk_freq);
+ return -EINVAL;
+ }
+
+ intel_state->cdclk.logical.vco = vco;
+ intel_state->cdclk.logical.cdclk = cdclk;
+
+ if (!intel_state->active_crtcs) {
+ cdclk = skl_calc_cdclk(0, vco);
+
+ intel_state->cdclk.actual.vco = vco;
+ intel_state->cdclk.actual.cdclk = cdclk;
+ } else {
+ intel_state->cdclk.actual =
+ intel_state->cdclk.logical;
+ }
+
+ return 0;
+}
+
+static int bxt_modeset_calc_cdclk(struct drm_atomic_state *state)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->dev);
+ int max_pixclk = intel_max_pixel_rate(state);
+ struct intel_atomic_state *intel_state =
+ to_intel_atomic_state(state);
+ int cdclk, vco;
+
+ if (IS_GEMINILAKE(dev_priv)) {
+ cdclk = glk_calc_cdclk(max_pixclk);
+ vco = glk_de_pll_vco(dev_priv, cdclk);
+ } else {
+ cdclk = bxt_calc_cdclk(max_pixclk);
+ vco = bxt_de_pll_vco(dev_priv, cdclk);
+ }
+
+ if (cdclk > dev_priv->max_cdclk_freq) {
+ DRM_DEBUG_KMS("requested cdclk (%d kHz) exceeds max (%d kHz)\n",
+ cdclk, dev_priv->max_cdclk_freq);
+ return -EINVAL;
+ }
+
+ intel_state->cdclk.logical.vco = vco;
+ intel_state->cdclk.logical.cdclk = cdclk;
+
+ if (!intel_state->active_crtcs) {
+ if (IS_GEMINILAKE(dev_priv)) {
+ cdclk = glk_calc_cdclk(0);
+ vco = glk_de_pll_vco(dev_priv, cdclk);
+ } else {
+ cdclk = bxt_calc_cdclk(0);
+ vco = bxt_de_pll_vco(dev_priv, cdclk);
+ }
+
+ intel_state->cdclk.actual.vco = vco;
+ intel_state->cdclk.actual.cdclk = cdclk;
+ } else {
+ intel_state->cdclk.actual =
+ intel_state->cdclk.logical;
+ }
+
+ return 0;
+}
+
+static int intel_compute_max_dotclk(struct drm_i915_private *dev_priv)
+{
+ int max_cdclk_freq = dev_priv->max_cdclk_freq;
+
+ if (IS_GEMINILAKE(dev_priv))
+ return 2 * max_cdclk_freq;
+ else if (INTEL_INFO(dev_priv)->gen >= 9 ||
+ IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
+ return max_cdclk_freq;
+ else if (IS_CHERRYVIEW(dev_priv))
+ return max_cdclk_freq*95/100;
+ else if (INTEL_INFO(dev_priv)->gen < 4)
+ return 2*max_cdclk_freq*90/100;
+ else
+ return max_cdclk_freq*90/100;
+}
+
+/**
+ * intel_update_max_cdclk - Determine the maximum support CDCLK frequency
+ * @dev_priv: i915 device
+ *
+ * Determine the maximum CDCLK frequency the platform supports, and also
+ * derive the maximum dot clock frequency the maximum CDCLK frequency
+ * allows.
+ */
+void intel_update_max_cdclk(struct drm_i915_private *dev_priv)
+{
+ if (IS_GEN9_BC(dev_priv)) {
+ u32 limit = I915_READ(SKL_DFSM) & SKL_DFSM_CDCLK_LIMIT_MASK;
+ int max_cdclk, vco;
+
+ vco = dev_priv->skl_preferred_vco_freq;
+ WARN_ON(vco != 8100000 && vco != 8640000);
+
+ /*
+ * Use the lower (vco 8640) cdclk values as a
+ * first guess. skl_calc_cdclk() will correct it
+ * if the preferred vco is 8100 instead.
+ */
+ if (limit == SKL_DFSM_CDCLK_LIMIT_675)
+ max_cdclk = 617143;
+ else if (limit == SKL_DFSM_CDCLK_LIMIT_540)
+ max_cdclk = 540000;
+ else if (limit == SKL_DFSM_CDCLK_LIMIT_450)
+ max_cdclk = 432000;
+ else
+ max_cdclk = 308571;
+
+ dev_priv->max_cdclk_freq = skl_calc_cdclk(max_cdclk, vco);
+ } else if (IS_GEMINILAKE(dev_priv)) {
+ dev_priv->max_cdclk_freq = 316800;
+ } else if (IS_BROXTON(dev_priv)) {
+ dev_priv->max_cdclk_freq = 624000;
+ } else if (IS_BROADWELL(dev_priv)) {
+ /*
+ * FIXME with extra cooling we can allow
+ * 540 MHz for ULX and 675 Mhz for ULT.
+ * How can we know if extra cooling is
+ * available? PCI ID, VTB, something else?
+ */
+ if (I915_READ(FUSE_STRAP) & HSW_CDCLK_LIMIT)
+ dev_priv->max_cdclk_freq = 450000;
+ else if (IS_BDW_ULX(dev_priv))
+ dev_priv->max_cdclk_freq = 450000;
+ else if (IS_BDW_ULT(dev_priv))
+ dev_priv->max_cdclk_freq = 540000;
+ else
+ dev_priv->max_cdclk_freq = 675000;
+ } else if (IS_CHERRYVIEW(dev_priv)) {
+ dev_priv->max_cdclk_freq = 320000;
+ } else if (IS_VALLEYVIEW(dev_priv)) {
+ dev_priv->max_cdclk_freq = 400000;
+ } else {
+ /* otherwise assume cdclk is fixed */
+ dev_priv->max_cdclk_freq = dev_priv->cdclk.hw.cdclk;
+ }
+
+ dev_priv->max_dotclk_freq = intel_compute_max_dotclk(dev_priv);
+
+ DRM_DEBUG_DRIVER("Max CD clock rate: %d kHz\n",
+ dev_priv->max_cdclk_freq);
+
+ DRM_DEBUG_DRIVER("Max dotclock rate: %d kHz\n",
+ dev_priv->max_dotclk_freq);
+}
+
+/**
+ * intel_update_cdclk - Determine the current CDCLK frequency
+ * @dev_priv: i915 device
+ *
+ * Determine the current CDCLK frequency.
+ */
+void intel_update_cdclk(struct drm_i915_private *dev_priv)
+{
+ dev_priv->display.get_cdclk(dev_priv, &dev_priv->cdclk.hw);
+
+ DRM_DEBUG_DRIVER("Current CD clock rate: %d kHz, VCO: %d kHz, ref: %d kHz\n",
+ dev_priv->cdclk.hw.cdclk, dev_priv->cdclk.hw.vco,
+ dev_priv->cdclk.hw.ref);
+
+ /*
+ * 9:0 CMBUS [sic] CDCLK frequency (cdfreq):
+ * Programmng [sic] note: bit[9:2] should be programmed to the number
+ * of cdclk that generates 4MHz reference clock freq which is used to
+ * generate GMBus clock. This will vary with the cdclk freq.
+ */
+ if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ I915_WRITE(GMBUSFREQ_VLV,
+ DIV_ROUND_UP(dev_priv->cdclk.hw.cdclk, 1000));
+}
+
+static int pch_rawclk(struct drm_i915_private *dev_priv)
+{
+ return (I915_READ(PCH_RAWCLK_FREQ) & RAWCLK_FREQ_MASK) * 1000;
+}
+
+static int vlv_hrawclk(struct drm_i915_private *dev_priv)
+{
+ /* RAWCLK_FREQ_VLV register updated from power well code */
+ return vlv_get_cck_clock_hpll(dev_priv, "hrawclk",
+ CCK_DISPLAY_REF_CLOCK_CONTROL);
+}
+
+static int g4x_hrawclk(struct drm_i915_private *dev_priv)
+{
+ uint32_t clkcfg;
+
+ /* hrawclock is 1/4 the FSB frequency */
+ clkcfg = I915_READ(CLKCFG);
+ switch (clkcfg & CLKCFG_FSB_MASK) {
+ case CLKCFG_FSB_400:
+ return 100000;
+ case CLKCFG_FSB_533:
+ return 133333;
+ case CLKCFG_FSB_667:
+ return 166667;
+ case CLKCFG_FSB_800:
+ return 200000;
+ case CLKCFG_FSB_1067:
+ return 266667;
+ case CLKCFG_FSB_1333:
+ return 333333;
+ /* these two are just a guess; one of them might be right */
+ case CLKCFG_FSB_1600:
+ case CLKCFG_FSB_1600_ALT:
+ return 400000;
+ default:
+ return 133333;
+ }
+}
+
+/**
+ * intel_update_rawclk - Determine the current RAWCLK frequency
+ * @dev_priv: i915 device
+ *
+ * Determine the current RAWCLK frequency. RAWCLK is a fixed
+ * frequency clock so this needs to done only once.
+ */
+void intel_update_rawclk(struct drm_i915_private *dev_priv)
+{
+ if (HAS_PCH_SPLIT(dev_priv))
+ dev_priv->rawclk_freq = pch_rawclk(dev_priv);
+ else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ dev_priv->rawclk_freq = vlv_hrawclk(dev_priv);
+ else if (IS_G4X(dev_priv) || IS_PINEVIEW(dev_priv))
+ dev_priv->rawclk_freq = g4x_hrawclk(dev_priv);
+ else
+ /* no rawclk on other platforms, or no need to know it */
+ return;
+
+ DRM_DEBUG_DRIVER("rawclk rate: %d kHz\n", dev_priv->rawclk_freq);
+}
+
+/**
+ * intel_init_cdclk_hooks - Initialize CDCLK related modesetting hooks
+ * @dev_priv: i915 device
+ */
+void intel_init_cdclk_hooks(struct drm_i915_private *dev_priv)
+{
+ if (IS_CHERRYVIEW(dev_priv)) {
+ dev_priv->display.set_cdclk = chv_set_cdclk;
+ dev_priv->display.modeset_calc_cdclk =
+ vlv_modeset_calc_cdclk;
+ } else if (IS_VALLEYVIEW(dev_priv)) {
+ dev_priv->display.set_cdclk = vlv_set_cdclk;
+ dev_priv->display.modeset_calc_cdclk =
+ vlv_modeset_calc_cdclk;
+ } else if (IS_BROADWELL(dev_priv)) {
+ dev_priv->display.set_cdclk = bdw_set_cdclk;
+ dev_priv->display.modeset_calc_cdclk =
+ bdw_modeset_calc_cdclk;
+ } else if (IS_GEN9_LP(dev_priv)) {
+ dev_priv->display.set_cdclk = bxt_set_cdclk;
+ dev_priv->display.modeset_calc_cdclk =
+ bxt_modeset_calc_cdclk;
+ } else if (IS_GEN9_BC(dev_priv)) {
+ dev_priv->display.set_cdclk = skl_set_cdclk;
+ dev_priv->display.modeset_calc_cdclk =
+ skl_modeset_calc_cdclk;
+ }
+
+ if (IS_GEN9_BC(dev_priv))
+ dev_priv->display.get_cdclk = skl_get_cdclk;
+ else if (IS_GEN9_LP(dev_priv))
+ dev_priv->display.get_cdclk = bxt_get_cdclk;
+ else if (IS_BROADWELL(dev_priv))
+ dev_priv->display.get_cdclk = bdw_get_cdclk;
+ else if (IS_HASWELL(dev_priv))
+ dev_priv->display.get_cdclk = hsw_get_cdclk;
+ else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ dev_priv->display.get_cdclk = vlv_get_cdclk;
+ else if (IS_GEN6(dev_priv) || IS_IVYBRIDGE(dev_priv))
+ dev_priv->display.get_cdclk = fixed_400mhz_get_cdclk;
+ else if (IS_GEN5(dev_priv))
+ dev_priv->display.get_cdclk = fixed_450mhz_get_cdclk;
+ else if (IS_GM45(dev_priv))
+ dev_priv->display.get_cdclk = gm45_get_cdclk;
+ else if (IS_G45(dev_priv))
+ dev_priv->display.get_cdclk = g33_get_cdclk;
+ else if (IS_I965GM(dev_priv))
+ dev_priv->display.get_cdclk = i965gm_get_cdclk;
+ else if (IS_I965G(dev_priv))
+ dev_priv->display.get_cdclk = fixed_400mhz_get_cdclk;
+ else if (IS_PINEVIEW(dev_priv))
+ dev_priv->display.get_cdclk = pnv_get_cdclk;
+ else if (IS_G33(dev_priv))
+ dev_priv->display.get_cdclk = g33_get_cdclk;
+ else if (IS_I945GM(dev_priv))
+ dev_priv->display.get_cdclk = i945gm_get_cdclk;
+ else if (IS_I945G(dev_priv))
+ dev_priv->display.get_cdclk = fixed_400mhz_get_cdclk;
+ else if (IS_I915GM(dev_priv))
+ dev_priv->display.get_cdclk = i915gm_get_cdclk;
+ else if (IS_I915G(dev_priv))
+ dev_priv->display.get_cdclk = fixed_333mhz_get_cdclk;
+ else if (IS_I865G(dev_priv))
+ dev_priv->display.get_cdclk = fixed_266mhz_get_cdclk;
+ else if (IS_I85X(dev_priv))
+ dev_priv->display.get_cdclk = i85x_get_cdclk;
+ else if (IS_I845G(dev_priv))
+ dev_priv->display.get_cdclk = fixed_200mhz_get_cdclk;
+ else { /* 830 */
+ WARN(!IS_I830(dev_priv),
+ "Unknown platform. Assuming 133 MHz CDCLK\n");
+ dev_priv->display.get_cdclk = fixed_133mhz_get_cdclk;
+ }
+}
hsw_enable_ips(intel_crtc);
}
-/* Loads the palette/gamma unit for the CRTC on Broadwell+. */
-static void broadwell_load_luts(struct drm_crtc_state *state)
+static void bdw_load_degamma_lut(struct drm_crtc_state *state)
{
- struct drm_crtc *crtc = state->crtc;
- struct drm_i915_private *dev_priv = to_i915(crtc->dev);
- struct intel_crtc_state *intel_state = to_intel_crtc_state(state);
- enum pipe pipe = to_intel_crtc(crtc)->pipe;
+ struct drm_i915_private *dev_priv = to_i915(state->crtc->dev);
+ enum pipe pipe = to_intel_crtc(state->crtc)->pipe;
uint32_t i, lut_size = INTEL_INFO(dev_priv)->color.degamma_lut_size;
- if (crtc_state_is_legacy(state)) {
- haswell_load_luts(state);
- return;
- }
-
I915_WRITE(PREC_PAL_INDEX(pipe),
PAL_PREC_SPLIT_MODE | PAL_PREC_AUTO_INCREMENT);
(v << 20) | (v << 10) | v);
}
}
+}
+
+static void bdw_load_gamma_lut(struct drm_crtc_state *state, u32 offset)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->crtc->dev);
+ enum pipe pipe = to_intel_crtc(state->crtc)->pipe;
+ uint32_t i, lut_size = INTEL_INFO(dev_priv)->color.gamma_lut_size;
+
+ WARN_ON(offset & ~PAL_PREC_INDEX_VALUE_MASK);
+
+ I915_WRITE(PREC_PAL_INDEX(pipe),
+ (offset ? PAL_PREC_SPLIT_MODE : 0) |
+ PAL_PREC_AUTO_INCREMENT |
+ offset);
if (state->gamma_lut) {
struct drm_color_lut *lut =
I915_WRITE(PREC_PAL_GC_MAX(pipe, 1), (1 << 16) - 1);
I915_WRITE(PREC_PAL_GC_MAX(pipe, 2), (1 << 16) - 1);
}
+}
+
+/* Loads the palette/gamma unit for the CRTC on Broadwell+. */
+static void broadwell_load_luts(struct drm_crtc_state *state)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->crtc->dev);
+ struct intel_crtc_state *intel_state = to_intel_crtc_state(state);
+ enum pipe pipe = to_intel_crtc(state->crtc)->pipe;
+
+ if (crtc_state_is_legacy(state)) {
+ haswell_load_luts(state);
+ return;
+ }
+
+ bdw_load_degamma_lut(state);
+ bdw_load_gamma_lut(state,
+ INTEL_INFO(dev_priv)->color.degamma_lut_size);
intel_state->gamma_mode = GAMMA_MODE_MODE_SPLIT;
I915_WRITE(GAMMA_MODE(pipe), GAMMA_MODE_MODE_SPLIT);
I915_WRITE(PREC_PAL_INDEX(pipe), 0);
}
+static void glk_load_degamma_lut(struct drm_crtc_state *state)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->crtc->dev);
+ enum pipe pipe = to_intel_crtc(state->crtc)->pipe;
+ const uint32_t lut_size = 33;
+ uint32_t i;
+
+ /*
+ * When setting the auto-increment bit, the hardware seems to
+ * ignore the index bits, so we need to reset it to index 0
+ * separately.
+ */
+ I915_WRITE(PRE_CSC_GAMC_INDEX(pipe), 0);
+ I915_WRITE(PRE_CSC_GAMC_INDEX(pipe), PRE_CSC_GAMC_AUTO_INCREMENT);
+
+ /*
+ * FIXME: The pipe degamma table in geminilake doesn't support
+ * different values per channel, so this just loads a linear table.
+ */
+ for (i = 0; i < lut_size; i++) {
+ uint32_t v = (i * (1 << 16)) / (lut_size - 1);
+
+ I915_WRITE(PRE_CSC_GAMC_DATA(pipe), v);
+ }
+
+ /* Clamp values > 1.0. */
+ while (i++ < 35)
+ I915_WRITE(PRE_CSC_GAMC_DATA(pipe), (1 << 16));
+}
+
+static void glk_load_luts(struct drm_crtc_state *state)
+{
+ struct drm_crtc *crtc = state->crtc;
+ struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct intel_crtc_state *intel_state = to_intel_crtc_state(state);
+ enum pipe pipe = to_intel_crtc(crtc)->pipe;
+
+ glk_load_degamma_lut(state);
+
+ if (crtc_state_is_legacy(state)) {
+ haswell_load_luts(state);
+ return;
+ }
+
+ bdw_load_gamma_lut(state, 0);
+
+ intel_state->gamma_mode = GAMMA_MODE_MODE_10BIT;
+ I915_WRITE(GAMMA_MODE(pipe), GAMMA_MODE_MODE_10BIT);
+ POSTING_READ(GAMMA_MODE(pipe));
+}
+
/* Loads the palette/gamma unit for the CRTC on CherryView. */
static void cherryview_load_luts(struct drm_crtc_state *state)
{
} else if (IS_HASWELL(dev_priv)) {
dev_priv->display.load_csc_matrix = i9xx_load_csc_matrix;
dev_priv->display.load_luts = haswell_load_luts;
- } else if (IS_BROADWELL(dev_priv) || IS_SKYLAKE(dev_priv) ||
- IS_BROXTON(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ } else if (IS_BROADWELL(dev_priv) || IS_GEN9_BC(dev_priv) ||
+ IS_BROXTON(dev_priv)) {
dev_priv->display.load_csc_matrix = i9xx_load_csc_matrix;
dev_priv->display.load_luts = broadwell_load_luts;
+ } else if (IS_GEMINILAKE(dev_priv)) {
+ dev_priv->display.load_csc_matrix = i9xx_load_csc_matrix;
+ dev_priv->display.load_luts = glk_load_luts;
} else {
dev_priv->display.load_luts = i9xx_load_luts;
}
struct drm_device *dev = encoder->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crt *crt = intel_encoder_to_crt(encoder);
- enum intel_display_power_domain power_domain;
u32 tmp;
bool ret;
- power_domain = intel_display_port_power_domain(encoder);
- if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
+ if (!intel_display_power_get_if_enabled(dev_priv,
+ encoder->power_domain))
return false;
ret = false;
ret = true;
out:
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, encoder->power_domain);
return ret;
}
struct drm_i915_private *dev_priv = to_i915(connector->dev);
struct intel_crt *crt = intel_attached_crt(connector);
struct intel_encoder *intel_encoder = &crt->base;
- enum intel_display_power_domain power_domain;
enum drm_connector_status status;
struct intel_load_detect_pipe tmp;
struct drm_modeset_acquire_ctx ctx;
if (dmi_check_system(intel_spurious_crt_detect))
return connector_status_disconnected;
- power_domain = intel_display_port_power_domain(intel_encoder);
- intel_display_power_get(dev_priv, power_domain);
+ intel_display_power_get(dev_priv, intel_encoder->power_domain);
if (I915_HAS_HOTPLUG(dev_priv)) {
/* We can not rely on the HPD pin always being correctly wired
drm_modeset_acquire_fini(&ctx);
out:
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, intel_encoder->power_domain);
return status;
}
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crt *crt = intel_attached_crt(connector);
struct intel_encoder *intel_encoder = &crt->base;
- enum intel_display_power_domain power_domain;
int ret;
struct i2c_adapter *i2c;
- power_domain = intel_display_port_power_domain(intel_encoder);
- intel_display_power_get(dev_priv, power_domain);
+ intel_display_power_get(dev_priv, intel_encoder->power_domain);
i2c = intel_gmbus_get_adapter(dev_priv, dev_priv->vbt.crt_ddc_pin);
ret = intel_crt_ddc_get_modes(connector, i2c);
ret = intel_crt_ddc_get_modes(connector, i2c);
out:
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, intel_encoder->power_domain);
return ret;
}
crt->adpa_reg = adpa_reg;
+ crt->base.power_domain = POWER_DOMAIN_PORT_CRT;
+
crt->base.compute_config = intel_crt_compute_config;
if (HAS_PCH_SPLIT(dev_priv)) {
crt->base.disable = pch_disable_crt;
* low-power state and comes back to normal.
*/
-#define I915_CSR_GLK "i915/glk_dmc_ver1_01.bin"
-#define GLK_CSR_VERSION_REQUIRED CSR_VERSION(1, 1)
+#define I915_CSR_GLK "i915/glk_dmc_ver1_04.bin"
+#define GLK_CSR_VERSION_REQUIRED CSR_VERSION(1, 4)
#define I915_CSR_KBL "i915/kbl_dmc_ver1_01.bin"
MODULE_FIRMWARE(I915_CSR_KBL);
struct drm_i915_private *dev_priv;
struct intel_csr *csr;
const struct firmware *fw = NULL;
- int ret;
dev_priv = container_of(work, typeof(*dev_priv), csr.work);
csr = &dev_priv->csr;
- ret = request_firmware(&fw, dev_priv->csr.fw_path,
- &dev_priv->drm.pdev->dev);
+ request_firmware(&fw, dev_priv->csr.fw_path, &dev_priv->drm.pdev->dev);
if (fw)
dev_priv->csr.dmc_payload = parse_csr_fw(dev_priv, fw);
u8 i_boost; /* SKL: I_boost; valid: 0x0, 0x1, 0x3, 0x7 */
};
+static const u8 index_to_dp_signal_levels[] = {
+ [0] = DP_TRAIN_VOLTAGE_SWING_LEVEL_0 | DP_TRAIN_PRE_EMPH_LEVEL_0,
+ [1] = DP_TRAIN_VOLTAGE_SWING_LEVEL_0 | DP_TRAIN_PRE_EMPH_LEVEL_1,
+ [2] = DP_TRAIN_VOLTAGE_SWING_LEVEL_0 | DP_TRAIN_PRE_EMPH_LEVEL_2,
+ [3] = DP_TRAIN_VOLTAGE_SWING_LEVEL_0 | DP_TRAIN_PRE_EMPH_LEVEL_3,
+ [4] = DP_TRAIN_VOLTAGE_SWING_LEVEL_1 | DP_TRAIN_PRE_EMPH_LEVEL_0,
+ [5] = DP_TRAIN_VOLTAGE_SWING_LEVEL_1 | DP_TRAIN_PRE_EMPH_LEVEL_1,
+ [6] = DP_TRAIN_VOLTAGE_SWING_LEVEL_1 | DP_TRAIN_PRE_EMPH_LEVEL_2,
+ [7] = DP_TRAIN_VOLTAGE_SWING_LEVEL_2 | DP_TRAIN_PRE_EMPH_LEVEL_0,
+ [8] = DP_TRAIN_VOLTAGE_SWING_LEVEL_2 | DP_TRAIN_PRE_EMPH_LEVEL_1,
+ [9] = DP_TRAIN_VOLTAGE_SWING_LEVEL_3 | DP_TRAIN_PRE_EMPH_LEVEL_0,
+};
+
/* HDMI/DVI modes ignore everything but the last 2 items. So we share
* them for both DP and FDI transports, allowing those ports to
* automatically adapt to HDMI connections as well
if (IS_GEN9_LP(dev_priv))
return hdmi_level;
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ if (IS_GEN9_BC(dev_priv)) {
skl_get_buf_trans_hdmi(dev_priv, &n_hdmi_entries);
hdmi_default_entry = 8;
} else if (IS_BROADWELL(dev_priv)) {
return hdmi_level;
}
+static const struct ddi_buf_trans *
+intel_ddi_get_buf_trans_dp(struct drm_i915_private *dev_priv,
+ int *n_entries)
+{
+ if (IS_KABYLAKE(dev_priv)) {
+ return kbl_get_buf_trans_dp(dev_priv, n_entries);
+ } else if (IS_SKYLAKE(dev_priv)) {
+ return skl_get_buf_trans_dp(dev_priv, n_entries);
+ } else if (IS_BROADWELL(dev_priv)) {
+ *n_entries = ARRAY_SIZE(bdw_ddi_translations_dp);
+ return bdw_ddi_translations_dp;
+ } else if (IS_HASWELL(dev_priv)) {
+ *n_entries = ARRAY_SIZE(hsw_ddi_translations_dp);
+ return hsw_ddi_translations_dp;
+ }
+
+ *n_entries = 0;
+ return NULL;
+}
+
+static const struct ddi_buf_trans *
+intel_ddi_get_buf_trans_edp(struct drm_i915_private *dev_priv,
+ int *n_entries)
+{
+ if (IS_KABYLAKE(dev_priv) || IS_SKYLAKE(dev_priv)) {
+ return skl_get_buf_trans_edp(dev_priv, n_entries);
+ } else if (IS_BROADWELL(dev_priv)) {
+ return bdw_get_buf_trans_edp(dev_priv, n_entries);
+ } else if (IS_HASWELL(dev_priv)) {
+ *n_entries = ARRAY_SIZE(hsw_ddi_translations_dp);
+ return hsw_ddi_translations_dp;
+ }
+
+ *n_entries = 0;
+ return NULL;
+}
+
+static const struct ddi_buf_trans *
+intel_ddi_get_buf_trans_fdi(struct drm_i915_private *dev_priv,
+ int *n_entries)
+{
+ if (IS_BROADWELL(dev_priv)) {
+ *n_entries = ARRAY_SIZE(hsw_ddi_translations_fdi);
+ return hsw_ddi_translations_fdi;
+ } else if (IS_HASWELL(dev_priv)) {
+ *n_entries = ARRAY_SIZE(hsw_ddi_translations_fdi);
+ return hsw_ddi_translations_fdi;
+ }
+
+ *n_entries = 0;
+ return NULL;
+}
+
/*
* Starting with Haswell, DDI port buffers must be programmed with correct
* values in advance. This function programs the correct values for
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
u32 iboost_bit = 0;
- int i, n_dp_entries, n_edp_entries, size;
+ int i, n_entries;
enum port port = intel_ddi_get_encoder_port(encoder);
- const struct ddi_buf_trans *ddi_translations_fdi;
- const struct ddi_buf_trans *ddi_translations_dp;
- const struct ddi_buf_trans *ddi_translations_edp;
const struct ddi_buf_trans *ddi_translations;
if (IS_GEN9_LP(dev_priv))
return;
- if (IS_KABYLAKE(dev_priv)) {
- ddi_translations_fdi = NULL;
- ddi_translations_dp =
- kbl_get_buf_trans_dp(dev_priv, &n_dp_entries);
- ddi_translations_edp =
- skl_get_buf_trans_edp(dev_priv, &n_edp_entries);
- } else if (IS_SKYLAKE(dev_priv)) {
- ddi_translations_fdi = NULL;
- ddi_translations_dp =
- skl_get_buf_trans_dp(dev_priv, &n_dp_entries);
- ddi_translations_edp =
- skl_get_buf_trans_edp(dev_priv, &n_edp_entries);
- } else if (IS_BROADWELL(dev_priv)) {
- ddi_translations_fdi = bdw_ddi_translations_fdi;
- ddi_translations_dp = bdw_ddi_translations_dp;
- ddi_translations_edp = bdw_get_buf_trans_edp(dev_priv, &n_edp_entries);
- n_dp_entries = ARRAY_SIZE(bdw_ddi_translations_dp);
- } else if (IS_HASWELL(dev_priv)) {
- ddi_translations_fdi = hsw_ddi_translations_fdi;
- ddi_translations_dp = hsw_ddi_translations_dp;
- ddi_translations_edp = hsw_ddi_translations_dp;
- n_dp_entries = n_edp_entries = ARRAY_SIZE(hsw_ddi_translations_dp);
- } else {
- WARN(1, "ddi translation table missing\n");
- ddi_translations_edp = bdw_ddi_translations_dp;
- ddi_translations_fdi = bdw_ddi_translations_fdi;
- ddi_translations_dp = bdw_ddi_translations_dp;
- n_edp_entries = ARRAY_SIZE(bdw_ddi_translations_edp);
- n_dp_entries = ARRAY_SIZE(bdw_ddi_translations_dp);
- }
-
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
- /* If we're boosting the current, set bit 31 of trans1 */
- if (dev_priv->vbt.ddi_port_info[port].dp_boost_level)
- iboost_bit = DDI_BUF_BALANCE_LEG_ENABLE;
-
- if (WARN_ON(encoder->type == INTEL_OUTPUT_EDP &&
- port != PORT_A && port != PORT_E &&
- n_edp_entries > 9))
- n_edp_entries = 9;
- }
-
switch (encoder->type) {
case INTEL_OUTPUT_EDP:
- ddi_translations = ddi_translations_edp;
- size = n_edp_entries;
+ ddi_translations = intel_ddi_get_buf_trans_edp(dev_priv,
+ &n_entries);
break;
case INTEL_OUTPUT_DP:
- ddi_translations = ddi_translations_dp;
- size = n_dp_entries;
+ ddi_translations = intel_ddi_get_buf_trans_dp(dev_priv,
+ &n_entries);
break;
case INTEL_OUTPUT_ANALOG:
- ddi_translations = ddi_translations_fdi;
- size = n_dp_entries;
+ ddi_translations = intel_ddi_get_buf_trans_fdi(dev_priv,
+ &n_entries);
break;
default:
- BUG();
+ MISSING_CASE(encoder->type);
+ return;
}
- for (i = 0; i < size; i++) {
+ if (IS_GEN9_BC(dev_priv)) {
+ /* If we're boosting the current, set bit 31 of trans1 */
+ if (dev_priv->vbt.ddi_port_info[port].dp_boost_level)
+ iboost_bit = DDI_BUF_BALANCE_LEG_ENABLE;
+
+ if (WARN_ON(encoder->type == INTEL_OUTPUT_EDP &&
+ port != PORT_A && port != PORT_E &&
+ n_entries > 9))
+ n_entries = 9;
+ }
+
+ for (i = 0; i < n_entries; i++) {
I915_WRITE(DDI_BUF_TRANS_LO(port, i),
ddi_translations[i].trans1 | iboost_bit);
I915_WRITE(DDI_BUF_TRANS_HI(port, i),
hdmi_level = intel_ddi_hdmi_level(dev_priv, port);
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ if (IS_GEN9_BC(dev_priv)) {
ddi_translations_hdmi = skl_get_buf_trans_hdmi(dev_priv, &n_hdmi_entries);
/* If we're boosting the current, set bit 31 of trans1 */
* DDI A (which is used for eDP)
*/
-void hsw_fdi_link_train(struct drm_crtc *crtc)
+void hsw_fdi_link_train(struct intel_crtc *crtc,
+ const struct intel_crtc_state *crtc_state)
{
- struct drm_device *dev = crtc->dev;
+ struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct intel_encoder *encoder;
u32 temp, i, rx_ctl_val, ddi_pll_sel;
- for_each_encoder_on_crtc(dev, crtc, encoder) {
+ for_each_encoder_on_crtc(dev, &crtc->base, encoder) {
WARN_ON(encoder->type != INTEL_OUTPUT_ANALOG);
intel_prepare_dp_ddi_buffers(encoder);
}
/* Enable the PCH Receiver FDI PLL */
rx_ctl_val = dev_priv->fdi_rx_config | FDI_RX_ENHANCE_FRAME_ENABLE |
FDI_RX_PLL_ENABLE |
- FDI_DP_PORT_WIDTH(intel_crtc->config->fdi_lanes);
+ FDI_DP_PORT_WIDTH(crtc_state->fdi_lanes);
I915_WRITE(FDI_RX_CTL(PIPE_A), rx_ctl_val);
POSTING_READ(FDI_RX_CTL(PIPE_A));
udelay(220);
I915_WRITE(FDI_RX_CTL(PIPE_A), rx_ctl_val);
/* Configure Port Clock Select */
- ddi_pll_sel = hsw_pll_to_ddi_pll_sel(intel_crtc->config->shared_dpll);
+ ddi_pll_sel = hsw_pll_to_ddi_pll_sel(crtc_state->shared_dpll);
I915_WRITE(PORT_CLK_SEL(PORT_E), ddi_pll_sel);
WARN_ON(ddi_pll_sel != PORT_CLK_SEL_SPLL);
* port reversal bit */
I915_WRITE(DDI_BUF_CTL(PORT_E),
DDI_BUF_CTL_ENABLE |
- ((intel_crtc->config->fdi_lanes - 1) << 1) |
+ ((crtc_state->fdi_lanes - 1) << 1) |
DDI_BUF_TRANS_SELECT(i / 2));
POSTING_READ(DDI_BUF_CTL(PORT_E));
}
static struct intel_encoder *
-intel_ddi_get_crtc_encoder(struct drm_crtc *crtc)
+intel_ddi_get_crtc_encoder(struct intel_crtc *crtc)
{
- struct drm_device *dev = crtc->dev;
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- struct intel_encoder *intel_encoder, *ret = NULL;
+ struct drm_device *dev = crtc->base.dev;
+ struct intel_encoder *encoder, *ret = NULL;
int num_encoders = 0;
- for_each_encoder_on_crtc(dev, crtc, intel_encoder) {
- ret = intel_encoder;
+ for_each_encoder_on_crtc(dev, &crtc->base, encoder) {
+ ret = encoder;
num_encoders++;
}
if (num_encoders != 1)
WARN(1, "%d encoders on crtc for pipe %c\n", num_encoders,
- pipe_name(intel_crtc->pipe));
+ pipe_name(crtc->pipe));
BUG_ON(ret == NULL);
return ret;
}
-struct intel_encoder *
+static struct intel_encoder *
intel_ddi_get_crtc_new_encoder(struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
state = crtc_state->base.state;
- for_each_connector_in_state(state, connector, connector_state, i) {
+ for_each_new_connector_in_state(state, connector, connector_state, i) {
if (connector_state->crtc != crtc_state->base.crtc)
continue;
if (INTEL_GEN(dev_priv) <= 8)
hsw_ddi_clock_get(encoder, pipe_config);
- else if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ else if (IS_GEN9_BC(dev_priv))
skl_ddi_clock_get(encoder, pipe_config);
else if (IS_GEN9_LP(dev_priv))
bxt_ddi_clock_get(encoder, pipe_config);
static bool
hsw_ddi_pll_select(struct intel_crtc *intel_crtc,
struct intel_crtc_state *crtc_state,
- struct intel_encoder *intel_encoder)
+ struct intel_encoder *encoder)
{
struct intel_shared_dpll *pll;
pll = intel_get_shared_dpll(intel_crtc, crtc_state,
- intel_encoder);
+ encoder);
if (!pll)
DRM_DEBUG_DRIVER("failed to find PLL for pipe %c\n",
pipe_name(intel_crtc->pipe));
static bool
skl_ddi_pll_select(struct intel_crtc *intel_crtc,
struct intel_crtc_state *crtc_state,
- struct intel_encoder *intel_encoder)
+ struct intel_encoder *encoder)
{
struct intel_shared_dpll *pll;
- pll = intel_get_shared_dpll(intel_crtc, crtc_state, intel_encoder);
+ pll = intel_get_shared_dpll(intel_crtc, crtc_state, encoder);
if (pll == NULL) {
DRM_DEBUG_DRIVER("failed to find PLL for pipe %c\n",
pipe_name(intel_crtc->pipe));
static bool
bxt_ddi_pll_select(struct intel_crtc *intel_crtc,
struct intel_crtc_state *crtc_state,
- struct intel_encoder *intel_encoder)
+ struct intel_encoder *encoder)
{
- return !!intel_get_shared_dpll(intel_crtc, crtc_state, intel_encoder);
+ return !!intel_get_shared_dpll(intel_crtc, crtc_state, encoder);
}
/*
struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(intel_crtc->base.dev);
- struct intel_encoder *intel_encoder =
+ struct intel_encoder *encoder =
intel_ddi_get_crtc_new_encoder(crtc_state);
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ if (IS_GEN9_BC(dev_priv))
return skl_ddi_pll_select(intel_crtc, crtc_state,
- intel_encoder);
+ encoder);
else if (IS_GEN9_LP(dev_priv))
return bxt_ddi_pll_select(intel_crtc, crtc_state,
- intel_encoder);
+ encoder);
else
return hsw_ddi_pll_select(intel_crtc, crtc_state,
- intel_encoder);
+ encoder);
}
-void intel_ddi_set_pipe_settings(struct drm_crtc *crtc)
+void intel_ddi_set_pipe_settings(const struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->dev);
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- struct intel_encoder *intel_encoder = intel_ddi_get_crtc_encoder(crtc);
- enum transcoder cpu_transcoder = intel_crtc->config->cpu_transcoder;
- int type = intel_encoder->type;
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct intel_encoder *encoder = intel_ddi_get_crtc_encoder(crtc);
+ enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
+ int type = encoder->type;
uint32_t temp;
if (type == INTEL_OUTPUT_DP || type == INTEL_OUTPUT_EDP || type == INTEL_OUTPUT_DP_MST) {
WARN_ON(transcoder_is_dsi(cpu_transcoder));
temp = TRANS_MSA_SYNC_CLK;
- switch (intel_crtc->config->pipe_bpp) {
+ switch (crtc_state->pipe_bpp) {
case 18:
temp |= TRANS_MSA_6_BPC;
break;
}
}
-void intel_ddi_set_vc_payload_alloc(struct drm_crtc *crtc, bool state)
+void intel_ddi_set_vc_payload_alloc(const struct intel_crtc_state *crtc_state,
+ bool state)
{
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- enum transcoder cpu_transcoder = intel_crtc->config->cpu_transcoder;
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
uint32_t temp;
temp = I915_READ(TRANS_DDI_FUNC_CTL(cpu_transcoder));
if (state == true)
I915_WRITE(TRANS_DDI_FUNC_CTL(cpu_transcoder), temp);
}
-void intel_ddi_enable_transcoder_func(struct drm_crtc *crtc)
+void intel_ddi_enable_transcoder_func(const struct intel_crtc_state *crtc_state)
{
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- struct intel_encoder *intel_encoder = intel_ddi_get_crtc_encoder(crtc);
- struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- enum pipe pipe = intel_crtc->pipe;
- enum transcoder cpu_transcoder = intel_crtc->config->cpu_transcoder;
- enum port port = intel_ddi_get_encoder_port(intel_encoder);
- int type = intel_encoder->type;
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+ struct intel_encoder *encoder = intel_ddi_get_crtc_encoder(crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum pipe pipe = crtc->pipe;
+ enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
+ enum port port = intel_ddi_get_encoder_port(encoder);
+ int type = encoder->type;
uint32_t temp;
/* Enable TRANS_DDI_FUNC_CTL for the pipe to work in HDMI mode */
temp = TRANS_DDI_FUNC_ENABLE;
temp |= TRANS_DDI_SELECT_PORT(port);
- switch (intel_crtc->config->pipe_bpp) {
+ switch (crtc_state->pipe_bpp) {
case 18:
temp |= TRANS_DDI_BPC_6;
break;
BUG();
}
- if (intel_crtc->config->base.adjusted_mode.flags & DRM_MODE_FLAG_PVSYNC)
+ if (crtc_state->base.adjusted_mode.flags & DRM_MODE_FLAG_PVSYNC)
temp |= TRANS_DDI_PVSYNC;
- if (intel_crtc->config->base.adjusted_mode.flags & DRM_MODE_FLAG_PHSYNC)
+ if (crtc_state->base.adjusted_mode.flags & DRM_MODE_FLAG_PHSYNC)
temp |= TRANS_DDI_PHSYNC;
if (cpu_transcoder == TRANSCODER_EDP) {
* using motion blur mitigation (which we don't
* support). */
if (IS_HASWELL(dev_priv) &&
- (intel_crtc->config->pch_pfit.enabled ||
- intel_crtc->config->pch_pfit.force_thru))
+ (crtc_state->pch_pfit.enabled ||
+ crtc_state->pch_pfit.force_thru))
temp |= TRANS_DDI_EDP_INPUT_A_ONOFF;
else
temp |= TRANS_DDI_EDP_INPUT_A_ON;
}
if (type == INTEL_OUTPUT_HDMI) {
- if (intel_crtc->config->has_hdmi_sink)
+ if (crtc_state->has_hdmi_sink)
temp |= TRANS_DDI_MODE_SELECT_HDMI;
else
temp |= TRANS_DDI_MODE_SELECT_DVI;
} else if (type == INTEL_OUTPUT_ANALOG) {
temp |= TRANS_DDI_MODE_SELECT_FDI;
- temp |= (intel_crtc->config->fdi_lanes - 1) << 1;
+ temp |= (crtc_state->fdi_lanes - 1) << 1;
} else if (type == INTEL_OUTPUT_DP ||
type == INTEL_OUTPUT_EDP) {
temp |= TRANS_DDI_MODE_SELECT_DP_SST;
- temp |= DDI_PORT_WIDTH(intel_crtc->config->lane_count);
+ temp |= DDI_PORT_WIDTH(crtc_state->lane_count);
} else if (type == INTEL_OUTPUT_DP_MST) {
temp |= TRANS_DDI_MODE_SELECT_DP_MST;
- temp |= DDI_PORT_WIDTH(intel_crtc->config->lane_count);
+ temp |= DDI_PORT_WIDTH(crtc_state->lane_count);
} else {
WARN(1, "Invalid encoder type %d for pipe %c\n",
- intel_encoder->type, pipe_name(pipe));
+ encoder->type, pipe_name(pipe));
}
I915_WRITE(TRANS_DDI_FUNC_CTL(cpu_transcoder), temp);
{
struct drm_device *dev = intel_connector->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_encoder *intel_encoder = intel_connector->encoder;
+ struct intel_encoder *encoder = intel_connector->encoder;
int type = intel_connector->base.connector_type;
- enum port port = intel_ddi_get_encoder_port(intel_encoder);
+ enum port port = intel_ddi_get_encoder_port(encoder);
enum pipe pipe = 0;
enum transcoder cpu_transcoder;
- enum intel_display_power_domain power_domain;
uint32_t tmp;
bool ret;
- power_domain = intel_display_port_power_domain(intel_encoder);
- if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
+ if (!intel_display_power_get_if_enabled(dev_priv,
+ encoder->power_domain))
return false;
- if (!intel_encoder->get_hw_state(intel_encoder, &pipe)) {
+ if (!encoder->get_hw_state(encoder, &pipe)) {
ret = false;
goto out;
}
}
out:
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, encoder->power_domain);
return ret;
}
struct drm_device *dev = encoder->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
enum port port = intel_ddi_get_encoder_port(encoder);
- enum intel_display_power_domain power_domain;
u32 tmp;
int i;
bool ret;
- power_domain = intel_display_port_power_domain(encoder);
- if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
+ if (!intel_display_power_get_if_enabled(dev_priv,
+ encoder->power_domain))
return false;
ret = false;
"(PHY_CTL %08x)\n", port_name(port), tmp);
}
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, encoder->power_domain);
return ret;
}
-void intel_ddi_enable_pipe_clock(struct intel_crtc *intel_crtc)
+static u64 intel_ddi_get_power_domains(struct intel_encoder *encoder)
{
- struct drm_crtc *crtc = &intel_crtc->base;
- struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_encoder *intel_encoder = intel_ddi_get_crtc_encoder(crtc);
- enum port port = intel_ddi_get_encoder_port(intel_encoder);
- enum transcoder cpu_transcoder = intel_crtc->config->cpu_transcoder;
+ struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
+ enum pipe pipe;
+
+ if (intel_ddi_get_hw_state(encoder, &pipe))
+ return BIT_ULL(dig_port->ddi_io_power_domain);
+
+ return 0;
+}
+
+void intel_ddi_enable_pipe_clock(const struct intel_crtc_state *crtc_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct intel_encoder *encoder = intel_ddi_get_crtc_encoder(crtc);
+ enum port port = intel_ddi_get_encoder_port(encoder);
+ enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
if (cpu_transcoder != TRANSCODER_EDP)
I915_WRITE(TRANS_CLK_SEL(cpu_transcoder),
TRANS_CLK_SEL_PORT(port));
}
-void intel_ddi_disable_pipe_clock(struct intel_crtc *intel_crtc)
+void intel_ddi_disable_pipe_clock(const struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(intel_crtc->base.dev);
- enum transcoder cpu_transcoder = intel_crtc->config->cpu_transcoder;
+ struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
+ enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
if (cpu_transcoder != TRANSCODER_EDP)
I915_WRITE(TRANS_CLK_SEL(cpu_transcoder),
ddi_translations[level].deemphasis);
}
-static uint32_t translate_signal_level(int signal_levels)
+u8 intel_ddi_dp_voltage_max(struct intel_encoder *encoder)
{
- uint32_t level;
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ int n_entries;
- switch (signal_levels) {
- default:
- DRM_DEBUG_KMS("Unsupported voltage swing/pre-emphasis level: 0x%x\n",
- signal_levels);
- case DP_TRAIN_VOLTAGE_SWING_LEVEL_0 | DP_TRAIN_PRE_EMPH_LEVEL_0:
- level = 0;
- break;
- case DP_TRAIN_VOLTAGE_SWING_LEVEL_0 | DP_TRAIN_PRE_EMPH_LEVEL_1:
- level = 1;
- break;
- case DP_TRAIN_VOLTAGE_SWING_LEVEL_0 | DP_TRAIN_PRE_EMPH_LEVEL_2:
- level = 2;
- break;
- case DP_TRAIN_VOLTAGE_SWING_LEVEL_0 | DP_TRAIN_PRE_EMPH_LEVEL_3:
- level = 3;
- break;
+ if (encoder->type == INTEL_OUTPUT_EDP)
+ intel_ddi_get_buf_trans_edp(dev_priv, &n_entries);
+ else
+ intel_ddi_get_buf_trans_dp(dev_priv, &n_entries);
- case DP_TRAIN_VOLTAGE_SWING_LEVEL_1 | DP_TRAIN_PRE_EMPH_LEVEL_0:
- level = 4;
- break;
- case DP_TRAIN_VOLTAGE_SWING_LEVEL_1 | DP_TRAIN_PRE_EMPH_LEVEL_1:
- level = 5;
- break;
- case DP_TRAIN_VOLTAGE_SWING_LEVEL_1 | DP_TRAIN_PRE_EMPH_LEVEL_2:
- level = 6;
- break;
+ if (WARN_ON(n_entries < 1))
+ n_entries = 1;
+ if (WARN_ON(n_entries > ARRAY_SIZE(index_to_dp_signal_levels)))
+ n_entries = ARRAY_SIZE(index_to_dp_signal_levels);
- case DP_TRAIN_VOLTAGE_SWING_LEVEL_2 | DP_TRAIN_PRE_EMPH_LEVEL_0:
- level = 7;
- break;
- case DP_TRAIN_VOLTAGE_SWING_LEVEL_2 | DP_TRAIN_PRE_EMPH_LEVEL_1:
- level = 8;
- break;
+ return index_to_dp_signal_levels[n_entries - 1] &
+ DP_TRAIN_VOLTAGE_SWING_MASK;
+}
- case DP_TRAIN_VOLTAGE_SWING_LEVEL_3 | DP_TRAIN_PRE_EMPH_LEVEL_0:
- level = 9;
- break;
+static uint32_t translate_signal_level(int signal_levels)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(index_to_dp_signal_levels); i++) {
+ if (index_to_dp_signal_levels[i] == signal_levels)
+ return i;
}
- return level;
+ WARN(1, "Unsupported voltage swing/pre-emphasis level: 0x%x\n",
+ signal_levels);
+
+ return 0;
}
uint32_t ddi_signal_levels(struct intel_dp *intel_dp)
level = translate_signal_level(signal_levels);
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ if (IS_GEN9_BC(dev_priv))
skl_ddi_set_iboost(encoder, level);
else if (IS_GEN9_LP(dev_priv))
bxt_ddi_vswing_sequence(dev_priv, level, port, encoder->type);
if (WARN_ON(!pll))
return;
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ if (IS_GEN9_BC(dev_priv)) {
uint32_t val;
/* DDI -> PLL mapping */
struct intel_dp *intel_dp = enc_to_intel_dp(&encoder->base);
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
enum port port = intel_ddi_get_encoder_port(encoder);
+ struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
+
+ WARN_ON(link_mst && (port == PORT_A || port == PORT_E));
intel_dp_set_link_params(intel_dp, link_rate, lane_count,
link_mst);
intel_edp_panel_on(intel_dp);
intel_ddi_clk_select(encoder, pll);
+
+ intel_display_power_get(dev_priv, dig_port->ddi_io_power_domain);
+
intel_prepare_dp_ddi_buffers(encoder);
intel_ddi_init_dp_buf_reg(encoder);
intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_ON);
struct drm_encoder *drm_encoder = &encoder->base;
enum port port = intel_ddi_get_encoder_port(encoder);
int level = intel_ddi_hdmi_level(dev_priv, port);
+ struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
intel_dp_dual_mode_set_tmds_output(intel_hdmi, true);
intel_ddi_clk_select(encoder, pll);
+
+ intel_display_power_get(dev_priv, dig_port->ddi_io_power_domain);
+
intel_prepare_hdmi_ddi_buffers(encoder);
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ if (IS_GEN9_BC(dev_priv))
skl_ddi_set_iboost(encoder, level);
else if (IS_GEN9_LP(dev_priv))
bxt_ddi_vswing_sequence(dev_priv, level, port,
crtc_state, conn_state);
}
-static void intel_ddi_pre_enable(struct intel_encoder *intel_encoder,
+static void intel_ddi_pre_enable(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config,
struct drm_connector_state *conn_state)
{
- struct drm_encoder *encoder = &intel_encoder->base;
- struct intel_crtc *crtc = to_intel_crtc(encoder->crtc);
- int type = intel_encoder->type;
+ int type = encoder->type;
if (type == INTEL_OUTPUT_DP || type == INTEL_OUTPUT_EDP) {
- intel_ddi_pre_enable_dp(intel_encoder,
- crtc->config->port_clock,
- crtc->config->lane_count,
- crtc->config->shared_dpll,
- intel_crtc_has_type(crtc->config,
+ intel_ddi_pre_enable_dp(encoder,
+ pipe_config->port_clock,
+ pipe_config->lane_count,
+ pipe_config->shared_dpll,
+ intel_crtc_has_type(pipe_config,
INTEL_OUTPUT_DP_MST));
}
if (type == INTEL_OUTPUT_HDMI) {
- intel_ddi_pre_enable_hdmi(intel_encoder,
+ intel_ddi_pre_enable_hdmi(encoder,
pipe_config->has_hdmi_sink,
pipe_config, conn_state,
- crtc->config->shared_dpll);
+ pipe_config->shared_dpll);
}
}
struct drm_encoder *encoder = &intel_encoder->base;
struct drm_i915_private *dev_priv = to_i915(encoder->dev);
enum port port = intel_ddi_get_encoder_port(intel_encoder);
+ struct intel_digital_port *dig_port = enc_to_dig_port(encoder);
int type = intel_encoder->type;
uint32_t val;
bool wait = false;
intel_edp_panel_off(intel_dp);
}
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ if (dig_port)
+ intel_display_power_put(dev_priv, dig_port->ddi_io_power_domain);
+
+ if (IS_GEN9_BC(dev_priv))
I915_WRITE(DPLL_CTRL2, (I915_READ(DPLL_CTRL2) |
DPLL_CTRL2_DDI_CLK_OFF(port)));
else if (INTEL_GEN(dev_priv) < 9)
}
}
-void intel_ddi_fdi_post_disable(struct intel_encoder *intel_encoder,
+void intel_ddi_fdi_post_disable(struct intel_encoder *encoder,
struct intel_crtc_state *old_crtc_state,
struct drm_connector_state *old_conn_state)
{
- struct drm_i915_private *dev_priv = to_i915(intel_encoder->base.dev);
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
uint32_t val;
/*
val &= ~FDI_RX_ENABLE;
I915_WRITE(FDI_RX_CTL(PIPE_A), val);
- intel_ddi_post_disable(intel_encoder, old_crtc_state, old_conn_state);
+ intel_ddi_post_disable(encoder, old_crtc_state, old_conn_state);
val = I915_READ(FDI_RX_MISC(PIPE_A));
val &= ~(FDI_RX_PWRDN_LANE1_MASK | FDI_RX_PWRDN_LANE0_MASK);
struct drm_connector_state *conn_state)
{
struct drm_encoder *encoder = &intel_encoder->base;
- struct drm_crtc *crtc = encoder->crtc;
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct drm_i915_private *dev_priv = to_i915(encoder->dev);
enum port port = intel_ddi_get_encoder_port(intel_encoder);
int type = intel_encoder->type;
intel_edp_drrs_enable(intel_dp, pipe_config);
}
- if (intel_crtc->config->has_audio) {
- intel_display_power_get(dev_priv, POWER_DOMAIN_AUDIO);
+ if (pipe_config->has_audio)
intel_audio_codec_enable(intel_encoder, pipe_config, conn_state);
- }
}
static void intel_disable_ddi(struct intel_encoder *intel_encoder,
struct drm_connector_state *old_conn_state)
{
struct drm_encoder *encoder = &intel_encoder->base;
- struct drm_crtc *crtc = encoder->crtc;
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
int type = intel_encoder->type;
- struct drm_device *dev = encoder->dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- if (intel_crtc->config->has_audio) {
+ if (old_crtc_state->has_audio)
intel_audio_codec_disable(intel_encoder);
- intel_display_power_put(dev_priv, POWER_DOMAIN_AUDIO);
- }
if (type == INTEL_OUTPUT_EDP) {
struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
struct intel_crtc_state *pipe_config,
struct drm_connector_state *conn_state)
{
- struct intel_crtc *intel_crtc = to_intel_crtc(encoder->base.crtc);
- uint8_t mask = intel_crtc->config->lane_lat_optim_mask;
+ uint8_t mask = pipe_config->lane_lat_optim_mask;
bxt_ddi_phy_set_lane_optim_mask(encoder, mask);
}
return connector;
}
-struct intel_shared_dpll *
-intel_ddi_get_link_dpll(struct intel_dp *intel_dp, int clock)
-{
- struct intel_connector *connector = intel_dp->attached_connector;
- struct intel_encoder *encoder = connector->encoder;
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
- struct intel_shared_dpll *pll = NULL;
- struct intel_shared_dpll_state tmp_pll_state;
- enum intel_dpll_id dpll_id;
-
- if (IS_GEN9_LP(dev_priv)) {
- dpll_id = (enum intel_dpll_id)dig_port->port;
- /*
- * Select the required PLL. This works for platforms where
- * there is no shared DPLL.
- */
- pll = &dev_priv->shared_dplls[dpll_id];
- if (WARN_ON(pll->active_mask)) {
-
- DRM_ERROR("Shared DPLL in use. active_mask:%x\n",
- pll->active_mask);
- return NULL;
- }
- tmp_pll_state = pll->state;
- if (!bxt_ddi_dp_set_dpll_hw_state(clock,
- &pll->state.hw_state)) {
- DRM_ERROR("Could not setup DPLL\n");
- pll->state = tmp_pll_state;
- return NULL;
- }
- } else if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
- pll = skl_find_link_pll(dev_priv, clock);
- } else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
- pll = hsw_ddi_dp_get_dpll(encoder, clock);
- }
- return pll;
-}
-
void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port)
{
struct intel_digital_port *intel_dig_port;
intel_encoder->get_hw_state = intel_ddi_get_hw_state;
intel_encoder->get_config = intel_ddi_get_config;
intel_encoder->suspend = intel_dp_encoder_suspend;
+ intel_encoder->get_power_domains = intel_ddi_get_power_domains;
intel_dig_port->port = port;
intel_dig_port->saved_port_bits = I915_READ(DDI_BUF_CTL(port)) &
(DDI_BUF_PORT_REVERSAL |
DDI_A_4_LANES);
+ switch (port) {
+ case PORT_A:
+ intel_dig_port->ddi_io_power_domain =
+ POWER_DOMAIN_PORT_DDI_A_IO;
+ break;
+ case PORT_B:
+ intel_dig_port->ddi_io_power_domain =
+ POWER_DOMAIN_PORT_DDI_B_IO;
+ break;
+ case PORT_C:
+ intel_dig_port->ddi_io_power_domain =
+ POWER_DOMAIN_PORT_DDI_C_IO;
+ break;
+ case PORT_D:
+ intel_dig_port->ddi_io_power_domain =
+ POWER_DOMAIN_PORT_DDI_D_IO;
+ break;
+ case PORT_E:
+ intel_dig_port->ddi_io_power_domain =
+ POWER_DOMAIN_PORT_DDI_E_IO;
+ break;
+ default:
+ MISSING_CASE(port);
+ }
+
/*
* Bspec says that DDI_A_4_LANES is the only supported configuration
* for Broxton. Yet some BIOS fail to set this bit on port A if eDP
intel_dig_port->max_lanes = max_lanes;
intel_encoder->type = INTEL_OUTPUT_UNKNOWN;
+ intel_encoder->power_domain = intel_port_to_power_domain(port);
intel_encoder->port = port;
intel_encoder->crtc_mask = (1 << 0) | (1 << 1) | (1 << 2);
intel_encoder->cloneable = 0;
goto err;
intel_dig_port->hpd_pulse = intel_dp_hpd_pulse;
- /*
- * On BXT A0/A1, sw needs to activate DDIA HPD logic and
- * interrupts to check the external panel connection.
- */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1) && port == PORT_B)
- dev_priv->hotplug.irq_port[PORT_A] = intel_dig_port;
- else
- dev_priv->hotplug.irq_port[port] = intel_dig_port;
+ dev_priv->hotplug.irq_port[port] = intel_dig_port;
}
/* In theory we don't need the encoder->type check, but leave it just in
const char *intel_platform_name(enum intel_platform platform)
{
+ BUILD_BUG_ON(ARRAY_SIZE(platform_names) != INTEL_MAX_PLATFORMS);
+
if (WARN_ON_ONCE(platform >= ARRAY_SIZE(platform_names) ||
platform_names[platform] == NULL))
return "<unknown>";
IS_GEN9_LP(dev_priv) && sseu_subslice_total(sseu) > 1;
sseu->has_eu_pg = sseu->eu_per_subslice > 2;
- if (IS_BROXTON(dev_priv)) {
+ if (IS_GEN9_LP(dev_priv)) {
#define IS_SS_DISABLED(ss) (!(sseu->subslice_mask & BIT(ss)))
+ info->has_pooled_eu = hweight8(sseu->subslice_mask) == 3;
+
/*
* There is a HW issue in 2x6 fused down parts that requires
* Pooled EU to be enabled as a WA. The pool configuration
* doesn't affect if the device has all 3 subslices enabled.
*/
/* WaEnablePooledEuFor2x6:bxt */
- info->has_pooled_eu = ((hweight8(sseu->subslice_mask) == 3) ||
- (hweight8(sseu->subslice_mask) == 2 &&
- INTEL_REVID(dev_priv) < BXT_REVID_C0));
+ info->has_pooled_eu |= (hweight8(sseu->subslice_mask) == 2 &&
+ IS_BXT_REVID(dev_priv, 0, BXT_REVID_B_LAST));
sseu->min_eu_in_pool = 0;
if (info->has_pooled_eu) {
* The subslice disable field is global, i.e. it applies
* to each of the enabled slices.
*/
- sseu->subslice_mask = BIT(ss_max) - 1;
+ sseu->subslice_mask = GENMASK(ss_max - 1, 0);
sseu->subslice_mask &= ~((fuse2 & GEN8_F2_SS_DIS_MASK) >>
GEN8_F2_SS_DIS_SHIFT);
info->has_snoop = !info->has_llc;
- /* Snooping is broken on BXT A stepping. */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1))
- info->has_snoop = false;
-
DRM_DEBUG_DRIVER("slice mask: %04x\n", info->sseu.slice_mask);
DRM_DEBUG_DRIVER("slice total: %u\n", hweight8(info->sseu.slice_mask));
DRM_DEBUG_DRIVER("subslice total: %u\n",
#include "intel_frontbuffer.h"
#include <drm/i915_drm.h>
#include "i915_drv.h"
+#include "i915_gem_clflush.h"
#include "intel_dsi.h"
#include "i915_trace.h"
#include <drm/drm_atomic.h>
static void ironlake_pch_clock_get(struct intel_crtc *crtc,
struct intel_crtc_state *pipe_config);
-static int intel_framebuffer_init(struct drm_device *dev,
- struct intel_framebuffer *ifb,
- struct drm_mode_fb_cmd2 *mode_cmd,
- struct drm_i915_gem_object *obj);
+static int intel_framebuffer_init(struct intel_framebuffer *ifb,
+ struct drm_i915_gem_object *obj,
+ struct drm_mode_fb_cmd2 *mode_cmd);
static void i9xx_set_pipeconf(struct intel_crtc *intel_crtc);
static void intel_set_pipe_timings(struct intel_crtc *intel_crtc);
static void intel_set_pipe_src_size(struct intel_crtc *intel_crtc);
static void ironlake_pfit_enable(struct intel_crtc *crtc);
static void intel_modeset_setup_hw_state(struct drm_device *dev);
static void intel_pre_disable_primary_noatomic(struct drm_crtc *crtc);
-static int ilk_max_pixel_rate(struct drm_atomic_state *state);
-static int glk_calc_cdclk(int max_pixclk);
-static int bxt_calc_cdclk(int max_pixclk);
struct intel_limit {
struct {
};
/* returns HPLL frequency in kHz */
-static int valleyview_get_vco(struct drm_i915_private *dev_priv)
+int vlv_get_hpll_vco(struct drm_i915_private *dev_priv)
{
int hpll_freq, vco_freq[] = { 800, 1600, 2000, 2400 };
return DIV_ROUND_CLOSEST(ref_freq << 1, divider + 1);
}
-static int vlv_get_cck_clock_hpll(struct drm_i915_private *dev_priv,
- const char *name, u32 reg)
+int vlv_get_cck_clock_hpll(struct drm_i915_private *dev_priv,
+ const char *name, u32 reg)
{
if (dev_priv->hpll_freq == 0)
- dev_priv->hpll_freq = valleyview_get_vco(dev_priv);
+ dev_priv->hpll_freq = vlv_get_hpll_vco(dev_priv);
return vlv_get_cck_clock(dev_priv, name, reg,
dev_priv->hpll_freq);
}
-static int
-intel_pch_rawclk(struct drm_i915_private *dev_priv)
-{
- return (I915_READ(PCH_RAWCLK_FREQ) & RAWCLK_FREQ_MASK) * 1000;
-}
-
-static int
-intel_vlv_hrawclk(struct drm_i915_private *dev_priv)
-{
- /* RAWCLK_FREQ_VLV register updated from power well code */
- return vlv_get_cck_clock_hpll(dev_priv, "hrawclk",
- CCK_DISPLAY_REF_CLOCK_CONTROL);
-}
-
-static int
-intel_g4x_hrawclk(struct drm_i915_private *dev_priv)
-{
- uint32_t clkcfg;
-
- /* hrawclock is 1/4 the FSB frequency */
- clkcfg = I915_READ(CLKCFG);
- switch (clkcfg & CLKCFG_FSB_MASK) {
- case CLKCFG_FSB_400:
- return 100000;
- case CLKCFG_FSB_533:
- return 133333;
- case CLKCFG_FSB_667:
- return 166667;
- case CLKCFG_FSB_800:
- return 200000;
- case CLKCFG_FSB_1067:
- return 266667;
- case CLKCFG_FSB_1333:
- return 333333;
- /* these two are just a guess; one of them might be right */
- case CLKCFG_FSB_1600:
- case CLKCFG_FSB_1600_ALT:
- return 400000;
- default:
- return 133333;
- }
-}
-
-void intel_update_rawclk(struct drm_i915_private *dev_priv)
-{
- if (HAS_PCH_SPLIT(dev_priv))
- dev_priv->rawclk_freq = intel_pch_rawclk(dev_priv);
- else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
- dev_priv->rawclk_freq = intel_vlv_hrawclk(dev_priv);
- else if (IS_G4X(dev_priv) || IS_PINEVIEW(dev_priv))
- dev_priv->rawclk_freq = intel_g4x_hrawclk(dev_priv);
- else
- return; /* no rawclk on other platforms, or no need to know it */
-
- DRM_DEBUG_DRIVER("rawclk rate: %d kHz\n", dev_priv->rawclk_freq);
-}
-
static void intel_update_czclk(struct drm_i915_private *dev_priv)
{
if (!(IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)))
return IS_GEN2(dev_priv) ? 2048 : 4096;
}
-static unsigned int intel_tile_width_bytes(const struct drm_i915_private *dev_priv,
- uint64_t fb_modifier, unsigned int cpp)
+static unsigned int
+intel_tile_width_bytes(const struct drm_framebuffer *fb, int plane)
{
- switch (fb_modifier) {
+ struct drm_i915_private *dev_priv = to_i915(fb->dev);
+ unsigned int cpp = fb->format->cpp[plane];
+
+ switch (fb->modifier) {
case DRM_FORMAT_MOD_NONE:
return cpp;
case I915_FORMAT_MOD_X_TILED:
}
break;
default:
- MISSING_CASE(fb_modifier);
+ MISSING_CASE(fb->modifier);
return cpp;
}
}
-unsigned int intel_tile_height(const struct drm_i915_private *dev_priv,
- uint64_t fb_modifier, unsigned int cpp)
+static unsigned int
+intel_tile_height(const struct drm_framebuffer *fb, int plane)
{
- if (fb_modifier == DRM_FORMAT_MOD_NONE)
+ if (fb->modifier == DRM_FORMAT_MOD_NONE)
return 1;
else
- return intel_tile_size(dev_priv) /
- intel_tile_width_bytes(dev_priv, fb_modifier, cpp);
+ return intel_tile_size(to_i915(fb->dev)) /
+ intel_tile_width_bytes(fb, plane);
}
/* Return the tile dimensions in pixel units */
-static void intel_tile_dims(const struct drm_i915_private *dev_priv,
+static void intel_tile_dims(const struct drm_framebuffer *fb, int plane,
unsigned int *tile_width,
- unsigned int *tile_height,
- uint64_t fb_modifier,
- unsigned int cpp)
+ unsigned int *tile_height)
{
- unsigned int tile_width_bytes =
- intel_tile_width_bytes(dev_priv, fb_modifier, cpp);
+ unsigned int tile_width_bytes = intel_tile_width_bytes(fb, plane);
+ unsigned int cpp = fb->format->cpp[plane];
*tile_width = tile_width_bytes / cpp;
- *tile_height = intel_tile_size(dev_priv) / tile_width_bytes;
+ *tile_height = intel_tile_size(to_i915(fb->dev)) / tile_width_bytes;
}
unsigned int
-intel_fb_align_height(struct drm_device *dev, unsigned int height,
- uint32_t pixel_format, uint64_t fb_modifier)
+intel_fb_align_height(const struct drm_framebuffer *fb,
+ int plane, unsigned int height)
{
- unsigned int cpp = drm_format_plane_cpp(pixel_format, 0);
- unsigned int tile_height = intel_tile_height(to_i915(dev), fb_modifier, cpp);
+ unsigned int tile_height = intel_tile_height(fb, plane);
return ALIGN(height, tile_height);
}
return 0;
}
-static unsigned int intel_surf_alignment(const struct drm_i915_private *dev_priv,
- uint64_t fb_modifier)
+static unsigned int intel_surf_alignment(const struct drm_framebuffer *fb,
+ int plane)
{
- switch (fb_modifier) {
+ struct drm_i915_private *dev_priv = to_i915(fb->dev);
+
+ /* AUX_DIST needs only 4K alignment */
+ if (fb->format->format == DRM_FORMAT_NV12 && plane == 1)
+ return 4096;
+
+ switch (fb->modifier) {
case DRM_FORMAT_MOD_NONE:
return intel_linear_alignment(dev_priv);
case I915_FORMAT_MOD_X_TILED:
- if (INTEL_INFO(dev_priv)->gen >= 9)
+ if (INTEL_GEN(dev_priv) >= 9)
return 256 * 1024;
return 0;
case I915_FORMAT_MOD_Y_TILED:
case I915_FORMAT_MOD_Yf_TILED:
return 1 * 1024 * 1024;
default:
- MISSING_CASE(fb_modifier);
+ MISSING_CASE(fb->modifier);
return 0;
}
}
WARN_ON(!mutex_is_locked(&dev->struct_mutex));
- alignment = intel_surf_alignment(dev_priv, fb->modifier);
+ alignment = intel_surf_alignment(fb, 0);
intel_fill_fb_ggtt_view(&view, fb, rotation);
unsigned int pitch_tiles;
tile_size = intel_tile_size(dev_priv);
- intel_tile_dims(dev_priv, &tile_width, &tile_height,
- fb->modifier, cpp);
+ intel_tile_dims(fb, plane, &tile_width, &tile_height);
if (drm_rotation_90_or_270(rotation)) {
pitch_tiles = pitch / tile_height;
unsigned int tile_rows, tiles, pitch_tiles;
tile_size = intel_tile_size(dev_priv);
- intel_tile_dims(dev_priv, &tile_width, &tile_height,
- fb_modifier, cpp);
+ intel_tile_dims(fb, plane, &tile_width, &tile_height);
if (drm_rotation_90_or_270(rotation)) {
pitch_tiles = pitch / tile_height;
const struct drm_framebuffer *fb = state->base.fb;
unsigned int rotation = state->base.rotation;
int pitch = intel_fb_pitch(fb, plane, rotation);
- u32 alignment;
-
- /* AUX_DIST needs only 4K alignment */
- if (fb->format->format == DRM_FORMAT_NV12 && plane == 1)
- alignment = 4096;
- else
- alignment = intel_surf_alignment(dev_priv, fb->modifier);
+ u32 alignment = intel_surf_alignment(fb, plane);
return _intel_compute_tile_offset(dev_priv, x, y, fb, plane, pitch,
rotation, alignment);
*/
if (i915_gem_object_is_tiled(intel_fb->obj) &&
(x + width) * cpp > fb->pitches[i]) {
- DRM_DEBUG("bad fb plane %d offset: 0x%x\n",
- i, fb->offsets[i]);
+ DRM_DEBUG_KMS("bad fb plane %d offset: 0x%x\n",
+ i, fb->offsets[i]);
return -EINVAL;
}
intel_fb->normal[i].y = y;
offset = _intel_compute_tile_offset(dev_priv, &x, &y,
- fb, 0, fb->pitches[i],
+ fb, i, fb->pitches[i],
DRM_ROTATE_0, tile_size);
offset /= tile_size;
unsigned int pitch_tiles;
struct drm_rect r;
- intel_tile_dims(dev_priv, &tile_width, &tile_height,
- fb->modifier, cpp);
+ intel_tile_dims(fb, i, &tile_width, &tile_height);
rot_info->plane[i].offset = offset;
rot_info->plane[i].stride = DIV_ROUND_UP(fb->pitches[i], tile_width * cpp);
max_size = max(max_size, offset + size);
}
- if (max_size * tile_size > to_intel_framebuffer(fb)->obj->base.size) {
- DRM_DEBUG("fb too big for bo (need %u bytes, have %zu bytes)\n",
- max_size * tile_size, to_intel_framebuffer(fb)->obj->base.size);
+ if (max_size * tile_size > intel_fb->obj->base.size) {
+ DRM_DEBUG_KMS("fb too big for bo (need %u bytes, have %zu bytes)\n",
+ max_size * tile_size, intel_fb->obj->base.size);
return -EINVAL;
}
return false;
mutex_lock(&dev->struct_mutex);
-
obj = i915_gem_object_create_stolen_for_preallocated(dev_priv,
base_aligned,
base_aligned,
size_aligned);
- if (!obj) {
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&dev->struct_mutex);
+ if (!obj)
return false;
- }
if (plane_config->tiling == I915_TILING_X)
obj->tiling_and_stride = fb->pitches[0] | I915_TILING_X;
mode_cmd.modifier[0] = fb->modifier;
mode_cmd.flags = DRM_MODE_FB_MODIFIERS;
- if (intel_framebuffer_init(dev, to_intel_framebuffer(fb),
- &mode_cmd, obj)) {
+ if (intel_framebuffer_init(to_intel_framebuffer(fb), obj, &mode_cmd)) {
DRM_DEBUG_KMS("intel fb init failed\n");
goto out_unref_obj;
}
- mutex_unlock(&dev->struct_mutex);
DRM_DEBUG_KMS("initial plane fb obj %p\n", obj);
return true;
out_unref_obj:
i915_gem_object_put(obj);
- mutex_unlock(&dev->struct_mutex);
return false;
}
drm_framebuffer_reference(plane->state->fb);
}
+static void
+intel_set_plane_visible(struct intel_crtc_state *crtc_state,
+ struct intel_plane_state *plane_state,
+ bool visible)
+{
+ struct intel_plane *plane = to_intel_plane(plane_state->base.plane);
+
+ plane_state->base.visible = visible;
+
+ /* FIXME pre-g4x don't work like this */
+ if (visible) {
+ crtc_state->base.plane_mask |= BIT(drm_plane_index(&plane->base));
+ crtc_state->active_planes |= BIT(plane->id);
+ } else {
+ crtc_state->base.plane_mask &= ~BIT(drm_plane_index(&plane->base));
+ crtc_state->active_planes &= ~BIT(plane->id);
+ }
+
+ DRM_DEBUG_KMS("%s active planes 0x%x\n",
+ crtc_state->base.crtc->name,
+ crtc_state->active_planes);
+}
+
static void
intel_find_initial_plane_obj(struct intel_crtc *intel_crtc,
struct intel_initial_plane_config *plane_config)
* simplest solution is to just disable the primary plane now and
* pretend the BIOS never had it enabled.
*/
- plane_state->visible = false;
- crtc_state->plane_mask &= ~(1 << drm_plane_index(primary));
+ intel_set_plane_visible(to_intel_crtc_state(crtc_state),
+ to_intel_plane_state(plane_state),
+ false);
intel_pre_disable_primary_noatomic(&intel_crtc->base);
+ trace_intel_disable_plane(primary, intel_crtc);
intel_plane->disable_plane(primary, &intel_crtc->base);
return;
drm_framebuffer_reference(fb);
primary->fb = primary->state->fb = fb;
primary->crtc = primary->state->crtc = &intel_crtc->base;
- intel_crtc->base.state->plane_mask |= (1 << drm_plane_index(primary));
+
+ intel_set_plane_visible(to_intel_crtc_state(crtc_state),
+ to_intel_plane_state(plane_state),
+ true);
+
atomic_or(to_intel_plane(primary)->frontbuffer_bit,
&obj->frontbuffer_bits);
}
static int skl_check_main_surface(struct intel_plane_state *plane_state)
{
- const struct drm_i915_private *dev_priv = to_i915(plane_state->base.plane->dev);
const struct drm_framebuffer *fb = plane_state->base.fb;
unsigned int rotation = plane_state->base.rotation;
int x = plane_state->base.src.x1 >> 16;
intel_add_fb_offsets(&x, &y, plane_state, 0);
offset = intel_compute_tile_offset(&x, &y, plane_state, 0);
-
- alignment = intel_surf_alignment(dev_priv, fb->modifier);
+ alignment = intel_surf_alignment(fb, 0);
/*
* AUX surface offset is specified as the distance from the
unsigned int rotation = plane_state->base.rotation;
int x = plane_state->base.src.x1 >> 16;
int y = plane_state->base.src.y1 >> 16;
+ unsigned long irqflags;
dspcntr = DISPPLANE_GAMMA_ENABLE;
if (INTEL_GEN(dev_priv) < 4) {
if (intel_crtc->pipe == PIPE_B)
dspcntr |= DISPPLANE_SEL_PIPE_B;
-
- /* pipesrc and dspsize control the size that is scaled from,
- * which should always be the user's requested size.
- */
- I915_WRITE(DSPSIZE(plane),
- ((crtc_state->pipe_src_h - 1) << 16) |
- (crtc_state->pipe_src_w - 1));
- I915_WRITE(DSPPOS(plane), 0);
- } else if (IS_CHERRYVIEW(dev_priv) && plane == PLANE_B) {
- I915_WRITE(PRIMSIZE(plane),
- ((crtc_state->pipe_src_h - 1) << 16) |
- (crtc_state->pipe_src_w - 1));
- I915_WRITE(PRIMPOS(plane), 0);
- I915_WRITE(PRIMCNSTALPHA(plane), 0);
}
switch (fb->format->format) {
intel_crtc->adjusted_x = x;
intel_crtc->adjusted_y = y;
- I915_WRITE(reg, dspcntr);
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
+ if (INTEL_GEN(dev_priv) < 4) {
+ /* pipesrc and dspsize control the size that is scaled from,
+ * which should always be the user's requested size.
+ */
+ I915_WRITE_FW(DSPSIZE(plane),
+ ((crtc_state->pipe_src_h - 1) << 16) |
+ (crtc_state->pipe_src_w - 1));
+ I915_WRITE_FW(DSPPOS(plane), 0);
+ } else if (IS_CHERRYVIEW(dev_priv) && plane == PLANE_B) {
+ I915_WRITE_FW(PRIMSIZE(plane),
+ ((crtc_state->pipe_src_h - 1) << 16) |
+ (crtc_state->pipe_src_w - 1));
+ I915_WRITE_FW(PRIMPOS(plane), 0);
+ I915_WRITE_FW(PRIMCNSTALPHA(plane), 0);
+ }
- I915_WRITE(DSPSTRIDE(plane), fb->pitches[0]);
+ I915_WRITE_FW(reg, dspcntr);
+
+ I915_WRITE_FW(DSPSTRIDE(plane), fb->pitches[0]);
if (INTEL_GEN(dev_priv) >= 4) {
- I915_WRITE(DSPSURF(plane),
- intel_plane_ggtt_offset(plane_state) +
- intel_crtc->dspaddr_offset);
- I915_WRITE(DSPTILEOFF(plane), (y << 16) | x);
- I915_WRITE(DSPLINOFF(plane), linear_offset);
+ I915_WRITE_FW(DSPSURF(plane),
+ intel_plane_ggtt_offset(plane_state) +
+ intel_crtc->dspaddr_offset);
+ I915_WRITE_FW(DSPTILEOFF(plane), (y << 16) | x);
+ I915_WRITE_FW(DSPLINOFF(plane), linear_offset);
} else {
- I915_WRITE(DSPADDR(plane),
- intel_plane_ggtt_offset(plane_state) +
- intel_crtc->dspaddr_offset);
+ I915_WRITE_FW(DSPADDR(plane),
+ intel_plane_ggtt_offset(plane_state) +
+ intel_crtc->dspaddr_offset);
}
- POSTING_READ(reg);
+ POSTING_READ_FW(reg);
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void i9xx_disable_primary_plane(struct drm_plane *primary,
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
int plane = intel_crtc->plane;
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
- I915_WRITE(DSPCNTR(plane), 0);
+ I915_WRITE_FW(DSPCNTR(plane), 0);
if (INTEL_INFO(dev_priv)->gen >= 4)
- I915_WRITE(DSPSURF(plane), 0);
+ I915_WRITE_FW(DSPSURF(plane), 0);
else
- I915_WRITE(DSPADDR(plane), 0);
- POSTING_READ(DSPCNTR(plane));
+ I915_WRITE_FW(DSPADDR(plane), 0);
+ POSTING_READ_FW(DSPCNTR(plane));
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void ironlake_update_primary_plane(struct drm_plane *primary,
unsigned int rotation = plane_state->base.rotation;
int x = plane_state->base.src.x1 >> 16;
int y = plane_state->base.src.y1 >> 16;
+ unsigned long irqflags;
dspcntr = DISPPLANE_GAMMA_ENABLE;
dspcntr |= DISPLAY_PLANE_ENABLE;
intel_crtc->adjusted_x = x;
intel_crtc->adjusted_y = y;
- I915_WRITE(reg, dspcntr);
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
- I915_WRITE(DSPSTRIDE(plane), fb->pitches[0]);
- I915_WRITE(DSPSURF(plane),
- intel_plane_ggtt_offset(plane_state) +
- intel_crtc->dspaddr_offset);
+ I915_WRITE_FW(reg, dspcntr);
+
+ I915_WRITE_FW(DSPSTRIDE(plane), fb->pitches[0]);
+ I915_WRITE_FW(DSPSURF(plane),
+ intel_plane_ggtt_offset(plane_state) +
+ intel_crtc->dspaddr_offset);
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
- I915_WRITE(DSPOFFSET(plane), (y << 16) | x);
+ I915_WRITE_FW(DSPOFFSET(plane), (y << 16) | x);
} else {
- I915_WRITE(DSPTILEOFF(plane), (y << 16) | x);
- I915_WRITE(DSPLINOFF(plane), linear_offset);
+ I915_WRITE_FW(DSPTILEOFF(plane), (y << 16) | x);
+ I915_WRITE_FW(DSPLINOFF(plane), linear_offset);
}
- POSTING_READ(reg);
+ POSTING_READ_FW(reg);
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
-u32 intel_fb_stride_alignment(const struct drm_i915_private *dev_priv,
- uint64_t fb_modifier, uint32_t pixel_format)
+static u32
+intel_fb_stride_alignment(const struct drm_framebuffer *fb, int plane)
{
- if (fb_modifier == DRM_FORMAT_MOD_NONE) {
+ if (fb->modifier == DRM_FORMAT_MOD_NONE)
return 64;
- } else {
- int cpp = drm_format_plane_cpp(pixel_format, 0);
-
- return intel_tile_width_bytes(dev_priv, fb_modifier, cpp);
- }
+ else
+ return intel_tile_width_bytes(fb, plane);
}
static void skl_detach_scaler(struct intel_crtc *intel_crtc, int id)
u32 skl_plane_stride(const struct drm_framebuffer *fb, int plane,
unsigned int rotation)
{
- const struct drm_i915_private *dev_priv = to_i915(fb->dev);
- u32 stride = intel_fb_pitch(fb, plane, rotation);
+ u32 stride;
+
+ if (plane >= fb->format->num_planes)
+ return 0;
+
+ stride = intel_fb_pitch(fb, plane, rotation);
/*
* The stride is either expressed as a multiple of 64 bytes chunks for
* linear buffers or in number of tiles for tiled buffers.
*/
- if (drm_rotation_90_or_270(rotation)) {
- int cpp = fb->format->cpp[plane];
-
- stride /= intel_tile_height(dev_priv, fb->modifier, cpp);
- } else {
- stride /= intel_fb_stride_alignment(dev_priv, fb->modifier,
- fb->format->format);
- }
+ if (drm_rotation_90_or_270(rotation))
+ stride /= intel_tile_height(fb, plane);
+ else
+ stride /= intel_fb_stride_alignment(fb, plane);
return stride;
}
int dst_y = plane_state->base.dst.y1;
int dst_w = drm_rect_width(&plane_state->base.dst);
int dst_h = drm_rect_height(&plane_state->base.dst);
+ unsigned long irqflags;
- plane_ctl = PLANE_CTL_ENABLE |
- PLANE_CTL_PIPE_GAMMA_ENABLE |
- PLANE_CTL_PIPE_CSC_ENABLE;
+ plane_ctl = PLANE_CTL_ENABLE;
+
+ if (!IS_GEMINILAKE(dev_priv)) {
+ plane_ctl |=
+ PLANE_CTL_PIPE_GAMMA_ENABLE |
+ PLANE_CTL_PIPE_CSC_ENABLE |
+ PLANE_CTL_PLANE_GAMMA_DISABLE;
+ }
plane_ctl |= skl_plane_ctl_format(fb->format->format);
plane_ctl |= skl_plane_ctl_tiling(fb->modifier);
- plane_ctl |= PLANE_CTL_PLANE_GAMMA_DISABLE;
plane_ctl |= skl_plane_ctl_rotation(rotation);
/* Sizes are 0 based */
intel_crtc->adjusted_x = src_x;
intel_crtc->adjusted_y = src_y;
- I915_WRITE(PLANE_CTL(pipe, plane_id), plane_ctl);
- I915_WRITE(PLANE_OFFSET(pipe, plane_id), (src_y << 16) | src_x);
- I915_WRITE(PLANE_STRIDE(pipe, plane_id), stride);
- I915_WRITE(PLANE_SIZE(pipe, plane_id), (src_h << 16) | src_w);
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
+ if (IS_GEMINILAKE(dev_priv)) {
+ I915_WRITE_FW(PLANE_COLOR_CTL(pipe, plane_id),
+ PLANE_COLOR_PIPE_GAMMA_ENABLE |
+ PLANE_COLOR_PIPE_CSC_ENABLE |
+ PLANE_COLOR_PLANE_GAMMA_DISABLE);
+ }
+
+ I915_WRITE_FW(PLANE_CTL(pipe, plane_id), plane_ctl);
+ I915_WRITE_FW(PLANE_OFFSET(pipe, plane_id), (src_y << 16) | src_x);
+ I915_WRITE_FW(PLANE_STRIDE(pipe, plane_id), stride);
+ I915_WRITE_FW(PLANE_SIZE(pipe, plane_id), (src_h << 16) | src_w);
if (scaler_id >= 0) {
uint32_t ps_ctrl = 0;
WARN_ON(!dst_w || !dst_h);
ps_ctrl = PS_SCALER_EN | PS_PLANE_SEL(plane_id) |
crtc_state->scaler_state.scalers[scaler_id].mode;
- I915_WRITE(SKL_PS_CTRL(pipe, scaler_id), ps_ctrl);
- I915_WRITE(SKL_PS_PWR_GATE(pipe, scaler_id), 0);
- I915_WRITE(SKL_PS_WIN_POS(pipe, scaler_id), (dst_x << 16) | dst_y);
- I915_WRITE(SKL_PS_WIN_SZ(pipe, scaler_id), (dst_w << 16) | dst_h);
- I915_WRITE(PLANE_POS(pipe, plane_id), 0);
+ I915_WRITE_FW(SKL_PS_CTRL(pipe, scaler_id), ps_ctrl);
+ I915_WRITE_FW(SKL_PS_PWR_GATE(pipe, scaler_id), 0);
+ I915_WRITE_FW(SKL_PS_WIN_POS(pipe, scaler_id), (dst_x << 16) | dst_y);
+ I915_WRITE_FW(SKL_PS_WIN_SZ(pipe, scaler_id), (dst_w << 16) | dst_h);
+ I915_WRITE_FW(PLANE_POS(pipe, plane_id), 0);
} else {
- I915_WRITE(PLANE_POS(pipe, plane_id), (dst_y << 16) | dst_x);
+ I915_WRITE_FW(PLANE_POS(pipe, plane_id), (dst_y << 16) | dst_x);
}
- I915_WRITE(PLANE_SURF(pipe, plane_id),
- intel_plane_ggtt_offset(plane_state) + surf_addr);
+ I915_WRITE_FW(PLANE_SURF(pipe, plane_id),
+ intel_plane_ggtt_offset(plane_state) + surf_addr);
+
+ POSTING_READ_FW(PLANE_SURF(pipe, plane_id));
- POSTING_READ(PLANE_SURF(pipe, plane_id));
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void skylake_disable_primary_plane(struct drm_plane *primary,
struct drm_i915_private *dev_priv = to_i915(dev);
enum plane_id plane_id = to_intel_plane(primary)->id;
enum pipe pipe = to_intel_plane(primary)->pipe;
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
+ I915_WRITE_FW(PLANE_CTL(pipe, plane_id), 0);
+ I915_WRITE_FW(PLANE_SURF(pipe, plane_id), 0);
+ POSTING_READ_FW(PLANE_SURF(pipe, plane_id));
- I915_WRITE(PLANE_CTL(pipe, plane_id), 0);
- I915_WRITE(PLANE_SURF(pipe, plane_id), 0);
- POSTING_READ(PLANE_SURF(pipe, plane_id));
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
/* Assume fb object is pinned & idle & fenced and just update base pointers */
struct intel_plane_state *plane_state =
to_intel_plane_state(plane->base.state);
- if (plane_state->base.visible)
+ if (plane_state->base.visible) {
+ trace_intel_update_plane(&plane->base,
+ to_intel_crtc(crtc));
+
plane->update_plane(&plane->base,
to_intel_crtc_state(crtc->state),
plane_state);
+ }
}
}
static int
__intel_display_resume(struct drm_device *dev,
- struct drm_atomic_state *state)
+ struct drm_atomic_state *state,
+ struct drm_modeset_acquire_ctx *ctx)
{
struct drm_crtc_state *crtc_state;
struct drm_crtc *crtc;
if (!state)
return 0;
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ /*
+ * We've duplicated the state, pointers to the old state are invalid.
+ *
+ * Don't attempt to use the old state until we commit the duplicated state.
+ */
+ for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
/*
* Force recalculation even if we restore
* current state. With fast modeset this may not result
}
/* ignore any reset values/BIOS leftovers in the WM registers */
- to_intel_atomic_state(state)->skip_intermediate_wm = true;
+ if (!HAS_GMCH_DISPLAY(to_i915(dev)))
+ to_intel_atomic_state(state)->skip_intermediate_wm = true;
- ret = drm_atomic_commit(state);
+ ret = drm_atomic_helper_commit_duplicated_state(state, ctx);
WARN_ON(ret == -EDEADLK);
return ret;
*/
intel_update_primary_planes(dev);
} else {
- ret = __intel_display_resume(dev, state);
+ ret = __intel_display_resume(dev, state, ctx);
if (ret)
DRM_ERROR("Restoring old state failed with %i\n", ret);
}
dev_priv->display.hpd_irq_setup(dev_priv);
spin_unlock_irq(&dev_priv->irq_lock);
- ret = __intel_display_resume(dev, state);
+ ret = __intel_display_resume(dev, state, ctx);
if (ret)
DRM_ERROR("Restoring old state failed with %i\n", ret);
{
struct i915_gpu_error *error = &to_i915(crtc->base.dev)->gpu_error;
- if (i915_reset_in_progress(error))
+ if (i915_reset_backoff(error))
return true;
if (crtc->reset_count != i915_reset_count(error))
}
}
-static void intel_fdi_normal_train(struct drm_crtc *crtc)
+static void intel_fdi_normal_train(struct intel_crtc *crtc)
{
- struct drm_device *dev = crtc->dev;
+ struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- int pipe = intel_crtc->pipe;
+ int pipe = crtc->pipe;
i915_reg_t reg;
u32 temp;
}
/* The FDI link training functions for ILK/Ibexpeak. */
-static void ironlake_fdi_link_train(struct drm_crtc *crtc)
+static void ironlake_fdi_link_train(struct intel_crtc *crtc,
+ const struct intel_crtc_state *crtc_state)
{
- struct drm_device *dev = crtc->dev;
+ struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- int pipe = intel_crtc->pipe;
+ int pipe = crtc->pipe;
i915_reg_t reg;
u32 temp, tries;
reg = FDI_TX_CTL(pipe);
temp = I915_READ(reg);
temp &= ~FDI_DP_PORT_WIDTH_MASK;
- temp |= FDI_DP_PORT_WIDTH(intel_crtc->config->fdi_lanes);
+ temp |= FDI_DP_PORT_WIDTH(crtc_state->fdi_lanes);
temp &= ~FDI_LINK_TRAIN_NONE;
temp |= FDI_LINK_TRAIN_PATTERN_1;
I915_WRITE(reg, temp | FDI_TX_ENABLE);
};
/* The FDI link training functions for SNB/Cougarpoint. */
-static void gen6_fdi_link_train(struct drm_crtc *crtc)
+static void gen6_fdi_link_train(struct intel_crtc *crtc,
+ const struct intel_crtc_state *crtc_state)
{
- struct drm_device *dev = crtc->dev;
+ struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- int pipe = intel_crtc->pipe;
+ int pipe = crtc->pipe;
i915_reg_t reg;
u32 temp, i, retry;
reg = FDI_TX_CTL(pipe);
temp = I915_READ(reg);
temp &= ~FDI_DP_PORT_WIDTH_MASK;
- temp |= FDI_DP_PORT_WIDTH(intel_crtc->config->fdi_lanes);
+ temp |= FDI_DP_PORT_WIDTH(crtc_state->fdi_lanes);
temp &= ~FDI_LINK_TRAIN_NONE;
temp |= FDI_LINK_TRAIN_PATTERN_1;
temp &= ~FDI_LINK_TRAIN_VOL_EMP_MASK;
}
/* Manual link training for Ivy Bridge A0 parts */
-static void ivb_manual_fdi_link_train(struct drm_crtc *crtc)
+static void ivb_manual_fdi_link_train(struct intel_crtc *crtc,
+ const struct intel_crtc_state *crtc_state)
{
- struct drm_device *dev = crtc->dev;
+ struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- int pipe = intel_crtc->pipe;
+ int pipe = crtc->pipe;
i915_reg_t reg;
u32 temp, i, j;
reg = FDI_TX_CTL(pipe);
temp = I915_READ(reg);
temp &= ~FDI_DP_PORT_WIDTH_MASK;
- temp |= FDI_DP_PORT_WIDTH(intel_crtc->config->fdi_lanes);
+ temp |= FDI_DP_PORT_WIDTH(crtc_state->fdi_lanes);
temp |= FDI_LINK_TRAIN_PATTERN_1_IVB;
temp &= ~FDI_LINK_TRAIN_VOL_EMP_MASK;
temp |= snb_b_fdi_train_param[j/2];
}
/* Program iCLKIP clock to the desired frequency */
-static void lpt_program_iclkip(struct drm_crtc *crtc)
+static void lpt_program_iclkip(struct intel_crtc *crtc)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->dev);
- int clock = to_intel_crtc(crtc)->config->base.adjusted_mode.crtc_clock;
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ int clock = crtc->config->base.adjusted_mode.crtc_clock;
u32 divsel, phaseinc, auxdiv, phasedir = 0;
u32 temp;
/* Return which DP Port should be selected for Transcoder DP control */
static enum port
-intel_trans_dp_port_sel(struct drm_crtc *crtc)
+intel_trans_dp_port_sel(struct intel_crtc *crtc)
{
- struct drm_device *dev = crtc->dev;
+ struct drm_device *dev = crtc->base.dev;
struct intel_encoder *encoder;
- for_each_encoder_on_crtc(dev, crtc, encoder) {
+ for_each_encoder_on_crtc(dev, &crtc->base, encoder) {
if (encoder->type == INTEL_OUTPUT_DP ||
encoder->type == INTEL_OUTPUT_EDP)
return enc_to_dig_port(&encoder->base)->port;
* - DP transcoding bits
* - transcoder
*/
-static void ironlake_pch_enable(struct drm_crtc *crtc)
+static void ironlake_pch_enable(const struct intel_crtc_state *crtc_state)
{
- struct drm_device *dev = crtc->dev;
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+ struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- int pipe = intel_crtc->pipe;
+ int pipe = crtc->pipe;
u32 temp;
assert_pch_transcoder_disabled(dev_priv, pipe);
if (IS_IVYBRIDGE(dev_priv))
- ivybridge_update_fdi_bc_bifurcation(intel_crtc);
+ ivybridge_update_fdi_bc_bifurcation(crtc);
/* Write the TU size bits before fdi link training, so that error
* detection works. */
I915_READ(PIPE_DATA_M1(pipe)) & TU_SIZE_MASK);
/* For PCH output, training FDI link */
- dev_priv->display.fdi_link_train(crtc);
+ dev_priv->display.fdi_link_train(crtc, crtc_state);
/* We need to program the right clock selection before writing the pixel
* mutliplier into the DPLL. */
temp = I915_READ(PCH_DPLL_SEL);
temp |= TRANS_DPLL_ENABLE(pipe);
sel = TRANS_DPLLB_SEL(pipe);
- if (intel_crtc->config->shared_dpll ==
+ if (crtc_state->shared_dpll ==
intel_get_shared_dpll_by_id(dev_priv, DPLL_ID_PCH_PLL_B))
temp |= sel;
else
* Note that enable_shared_dpll tries to do the right thing, but
* get_shared_dpll unconditionally resets the pll - we need that to have
* the right LVDS enable sequence. */
- intel_enable_shared_dpll(intel_crtc);
+ intel_enable_shared_dpll(crtc);
/* set transcoder timing, panel must allow it */
assert_panel_unlocked(dev_priv, pipe);
- ironlake_pch_transcoder_set_timings(intel_crtc, pipe);
+ ironlake_pch_transcoder_set_timings(crtc, pipe);
intel_fdi_normal_train(crtc);
/* For PCH DP, enable TRANS_DP_CTL */
if (HAS_PCH_CPT(dev_priv) &&
- intel_crtc_has_dp_encoder(intel_crtc->config)) {
+ intel_crtc_has_dp_encoder(crtc_state)) {
const struct drm_display_mode *adjusted_mode =
- &intel_crtc->config->base.adjusted_mode;
+ &crtc_state->base.adjusted_mode;
u32 bpc = (I915_READ(PIPECONF(pipe)) & PIPECONF_BPC_MASK) >> 5;
i915_reg_t reg = TRANS_DP_CTL(pipe);
temp = I915_READ(reg);
ironlake_enable_pch_transcoder(dev_priv, pipe);
}
-static void lpt_pch_enable(struct drm_crtc *crtc)
+static void lpt_pch_enable(const struct intel_crtc_state *crtc_state)
{
- struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- enum transcoder cpu_transcoder = intel_crtc->config->cpu_transcoder;
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
assert_pch_transcoder_disabled(dev_priv, TRANSCODER_A);
lpt_program_iclkip(crtc);
/* Set transcoder timing. */
- ironlake_pch_transcoder_set_timings(intel_crtc, PIPE_A);
+ ironlake_pch_transcoder_set_timings(crtc, PIPE_A);
lpt_enable_pch_transcoder(dev_priv, cpu_transcoder);
}
intel_frontbuffer_flip(to_i915(crtc->base.dev), pipe_config->fb_bits);
- crtc->wm.cxsr_allowed = true;
-
if (pipe_config->update_wm_post && pipe_config->base.active)
intel_update_watermarks(crtc);
}
}
-static void intel_pre_plane_update(struct intel_crtc_state *old_crtc_state)
+static void intel_pre_plane_update(struct intel_crtc_state *old_crtc_state,
+ struct intel_crtc_state *pipe_config)
{
struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->base.crtc);
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_crtc_state *pipe_config =
- to_intel_crtc_state(crtc->base.state);
struct drm_atomic_state *old_state = old_crtc_state->base.state;
struct drm_plane *primary = crtc->base.primary;
struct drm_plane_state *old_pri_state =
intel_pre_disable_primary(&crtc->base);
}
- if (pipe_config->disable_cxsr && HAS_GMCH_DISPLAY(dev_priv)) {
- crtc->wm.cxsr_allowed = false;
-
- /*
- * Vblank time updates from the shadow to live plane control register
- * are blocked if the memory self-refresh mode is active at that
- * moment. So to make sure the plane gets truly disabled, disable
- * first the self-refresh mode. The self-refresh enable bit in turn
- * will be checked/applied by the HW only at the next frame start
- * event which is after the vblank start event, so we need to have a
- * wait-for-vblank between disabling the plane and the pipe.
- */
- if (old_crtc_state->base.active &&
- intel_set_memory_cxsr(dev_priv, false))
- intel_wait_for_vblank(dev_priv, crtc->pipe);
- }
+ /*
+ * Vblank time updates from the shadow to live plane control register
+ * are blocked if the memory self-refresh mode is active at that
+ * moment. So to make sure the plane gets truly disabled, disable
+ * first the self-refresh mode. The self-refresh enable bit in turn
+ * will be checked/applied by the HW only at the next frame start
+ * event which is after the vblank start event, so we need to have a
+ * wait-for-vblank between disabling the plane and the pipe.
+ */
+ if (HAS_GMCH_DISPLAY(dev_priv) && old_crtc_state->base.active &&
+ pipe_config->disable_cxsr && intel_set_memory_cxsr(dev_priv, false))
+ intel_wait_for_vblank(dev_priv, crtc->pipe);
/*
* IVB workaround: must disable low power watermarks for at least
struct intel_crtc_state *crtc_state,
struct drm_atomic_state *old_state)
{
- struct drm_connector_state *old_conn_state;
+ struct drm_connector_state *conn_state;
struct drm_connector *conn;
int i;
- for_each_connector_in_state(old_state, conn, old_conn_state, i) {
- struct drm_connector_state *conn_state = conn->state;
+ for_each_new_connector_in_state(old_state, conn, conn_state, i) {
struct intel_encoder *encoder =
to_intel_encoder(conn_state->best_encoder);
struct intel_crtc_state *crtc_state,
struct drm_atomic_state *old_state)
{
- struct drm_connector_state *old_conn_state;
+ struct drm_connector_state *conn_state;
struct drm_connector *conn;
int i;
- for_each_connector_in_state(old_state, conn, old_conn_state, i) {
- struct drm_connector_state *conn_state = conn->state;
+ for_each_new_connector_in_state(old_state, conn, conn_state, i) {
struct intel_encoder *encoder =
to_intel_encoder(conn_state->best_encoder);
struct intel_crtc_state *crtc_state,
struct drm_atomic_state *old_state)
{
- struct drm_connector_state *old_conn_state;
+ struct drm_connector_state *conn_state;
struct drm_connector *conn;
int i;
- for_each_connector_in_state(old_state, conn, old_conn_state, i) {
- struct drm_connector_state *conn_state = conn->state;
+ for_each_new_connector_in_state(old_state, conn, conn_state, i) {
struct intel_encoder *encoder =
to_intel_encoder(conn_state->best_encoder);
struct drm_connector *conn;
int i;
- for_each_connector_in_state(old_state, conn, old_conn_state, i) {
+ for_each_old_connector_in_state(old_state, conn, old_conn_state, i) {
struct intel_encoder *encoder =
to_intel_encoder(old_conn_state->best_encoder);
struct drm_connector *conn;
int i;
- for_each_connector_in_state(old_state, conn, old_conn_state, i) {
+ for_each_old_connector_in_state(old_state, conn, old_conn_state, i) {
struct intel_encoder *encoder =
to_intel_encoder(old_conn_state->best_encoder);
struct drm_connector *conn;
int i;
- for_each_connector_in_state(old_state, conn, old_conn_state, i) {
+ for_each_old_connector_in_state(old_state, conn, old_conn_state, i) {
struct intel_encoder *encoder =
to_intel_encoder(old_conn_state->best_encoder);
intel_enable_pipe(intel_crtc);
if (intel_crtc->config->has_pch_encoder)
- ironlake_pch_enable(crtc);
+ ironlake_pch_enable(pipe_config);
assert_vblank_disabled(crtc);
drm_crtc_vblank_on(crtc);
intel_encoders_pre_enable(crtc, pipe_config, old_state);
if (intel_crtc->config->has_pch_encoder)
- dev_priv->display.fdi_link_train(crtc);
+ dev_priv->display.fdi_link_train(intel_crtc, pipe_config);
if (!transcoder_is_dsi(cpu_transcoder))
- intel_ddi_enable_pipe_clock(intel_crtc);
+ intel_ddi_enable_pipe_clock(pipe_config);
if (INTEL_GEN(dev_priv) >= 9)
skylake_pfit_enable(intel_crtc);
*/
intel_color_load_luts(&pipe_config->base);
- intel_ddi_set_pipe_settings(crtc);
+ intel_ddi_set_pipe_settings(pipe_config);
if (!transcoder_is_dsi(cpu_transcoder))
- intel_ddi_enable_transcoder_func(crtc);
+ intel_ddi_enable_transcoder_func(pipe_config);
if (dev_priv->display.initial_watermarks != NULL)
dev_priv->display.initial_watermarks(old_intel_state, pipe_config);
intel_enable_pipe(intel_crtc);
if (intel_crtc->config->has_pch_encoder)
- lpt_pch_enable(crtc);
+ lpt_pch_enable(pipe_config);
if (intel_crtc_has_type(intel_crtc->config, INTEL_OUTPUT_DP_MST))
- intel_ddi_set_vc_payload_alloc(crtc, true);
+ intel_ddi_set_vc_payload_alloc(pipe_config, true);
assert_vblank_disabled(crtc);
drm_crtc_vblank_on(crtc);
intel_disable_pipe(intel_crtc);
if (intel_crtc_has_type(intel_crtc->config, INTEL_OUTPUT_DP_MST))
- intel_ddi_set_vc_payload_alloc(crtc, false);
+ intel_ddi_set_vc_payload_alloc(intel_crtc->config, false);
if (!transcoder_is_dsi(cpu_transcoder))
intel_ddi_disable_transcoder_func(dev_priv, cpu_transcoder);
ironlake_pfit_disable(intel_crtc, false);
if (!transcoder_is_dsi(cpu_transcoder))
- intel_ddi_disable_pipe_clock(intel_crtc);
+ intel_ddi_disable_pipe_clock(intel_crtc->config);
intel_encoders_post_disable(crtc, old_crtc_state, old_state);
I915_WRITE(BCLRPAT(crtc->pipe), 0);
}
-static enum intel_display_power_domain port_to_power_domain(enum port port)
+enum intel_display_power_domain intel_port_to_power_domain(enum port port)
{
switch (port) {
case PORT_A:
}
}
-static enum intel_display_power_domain port_to_aux_power_domain(enum port port)
-{
- switch (port) {
- case PORT_A:
- return POWER_DOMAIN_AUX_A;
- case PORT_B:
- return POWER_DOMAIN_AUX_B;
- case PORT_C:
- return POWER_DOMAIN_AUX_C;
- case PORT_D:
- return POWER_DOMAIN_AUX_D;
- case PORT_E:
- /* FIXME: Check VBT for actual wiring of PORT E */
- return POWER_DOMAIN_AUX_D;
- default:
- MISSING_CASE(port);
- return POWER_DOMAIN_AUX_A;
- }
-}
-
-enum intel_display_power_domain
-intel_display_port_power_domain(struct intel_encoder *intel_encoder)
-{
- struct drm_i915_private *dev_priv = to_i915(intel_encoder->base.dev);
- struct intel_digital_port *intel_dig_port;
-
- switch (intel_encoder->type) {
- case INTEL_OUTPUT_UNKNOWN:
- /* Only DDI platforms should ever use this output type */
- WARN_ON_ONCE(!HAS_DDI(dev_priv));
- case INTEL_OUTPUT_DP:
- case INTEL_OUTPUT_HDMI:
- case INTEL_OUTPUT_EDP:
- intel_dig_port = enc_to_dig_port(&intel_encoder->base);
- return port_to_power_domain(intel_dig_port->port);
- case INTEL_OUTPUT_DP_MST:
- intel_dig_port = enc_to_mst(&intel_encoder->base)->primary;
- return port_to_power_domain(intel_dig_port->port);
- case INTEL_OUTPUT_ANALOG:
- return POWER_DOMAIN_PORT_CRT;
- case INTEL_OUTPUT_DSI:
- return POWER_DOMAIN_PORT_DSI;
- default:
- return POWER_DOMAIN_PORT_OTHER;
- }
-}
-
-enum intel_display_power_domain
-intel_display_port_aux_power_domain(struct intel_encoder *intel_encoder)
-{
- struct drm_i915_private *dev_priv = to_i915(intel_encoder->base.dev);
- struct intel_digital_port *intel_dig_port;
-
- switch (intel_encoder->type) {
- case INTEL_OUTPUT_UNKNOWN:
- case INTEL_OUTPUT_HDMI:
- /*
- * Only DDI platforms should ever use these output types.
- * We can get here after the HDMI detect code has already set
- * the type of the shared encoder. Since we can't be sure
- * what's the status of the given connectors, play safe and
- * run the DP detection too.
- */
- WARN_ON_ONCE(!HAS_DDI(dev_priv));
- case INTEL_OUTPUT_DP:
- case INTEL_OUTPUT_EDP:
- intel_dig_port = enc_to_dig_port(&intel_encoder->base);
- return port_to_aux_power_domain(intel_dig_port->port);
- case INTEL_OUTPUT_DP_MST:
- intel_dig_port = enc_to_mst(&intel_encoder->base)->primary;
- return port_to_aux_power_domain(intel_dig_port->port);
- default:
- MISSING_CASE(intel_encoder->type);
- return POWER_DOMAIN_AUX_A;
- }
-}
-
-static unsigned long get_crtc_power_domains(struct drm_crtc *crtc,
- struct intel_crtc_state *crtc_state)
+static u64 get_crtc_power_domains(struct drm_crtc *crtc,
+ struct intel_crtc_state *crtc_state)
{
struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_encoder *encoder;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
enum pipe pipe = intel_crtc->pipe;
- unsigned long mask;
+ u64 mask;
enum transcoder transcoder = crtc_state->cpu_transcoder;
if (!crtc_state->base.active)
mask |= BIT(POWER_DOMAIN_TRANSCODER(transcoder));
if (crtc_state->pch_pfit.enabled ||
crtc_state->pch_pfit.force_thru)
- mask |= BIT(POWER_DOMAIN_PIPE_PANEL_FITTER(pipe));
+ mask |= BIT_ULL(POWER_DOMAIN_PIPE_PANEL_FITTER(pipe));
drm_for_each_encoder_mask(encoder, dev, crtc_state->base.encoder_mask) {
struct intel_encoder *intel_encoder = to_intel_encoder(encoder);
- mask |= BIT(intel_display_port_power_domain(intel_encoder));
+ mask |= BIT_ULL(intel_encoder->power_domain);
}
+ if (HAS_DDI(dev_priv) && crtc_state->has_audio)
+ mask |= BIT(POWER_DOMAIN_AUDIO);
+
if (crtc_state->shared_dpll)
- mask |= BIT(POWER_DOMAIN_PLLS);
+ mask |= BIT_ULL(POWER_DOMAIN_PLLS);
return mask;
}
-static unsigned long
+static u64
modeset_get_crtc_power_domains(struct drm_crtc *crtc,
struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(crtc->dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
enum intel_display_power_domain domain;
- unsigned long domains, new_domains, old_domains;
+ u64 domains, new_domains, old_domains;
old_domains = intel_crtc->enabled_power_domains;
intel_crtc->enabled_power_domains = new_domains =
}
static void modeset_put_power_domains(struct drm_i915_private *dev_priv,
- unsigned long domains)
+ u64 domains)
{
enum intel_display_power_domain domain;
intel_display_power_put(dev_priv, domain);
}
-static int intel_compute_max_dotclk(struct drm_i915_private *dev_priv)
+static void valleyview_crtc_enable(struct intel_crtc_state *pipe_config,
+ struct drm_atomic_state *old_state)
{
- int max_cdclk_freq = dev_priv->max_cdclk_freq;
+ struct intel_atomic_state *old_intel_state =
+ to_intel_atomic_state(old_state);
+ struct drm_crtc *crtc = pipe_config->base.crtc;
+ struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
+ int pipe = intel_crtc->pipe;
- if (IS_GEMINILAKE(dev_priv))
- return 2 * max_cdclk_freq;
- else if (INTEL_INFO(dev_priv)->gen >= 9 ||
- IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
- return max_cdclk_freq;
- else if (IS_CHERRYVIEW(dev_priv))
- return max_cdclk_freq*95/100;
- else if (INTEL_INFO(dev_priv)->gen < 4)
- return 2*max_cdclk_freq*90/100;
- else
- return max_cdclk_freq*90/100;
-}
+ if (WARN_ON(intel_crtc->active))
+ return;
-static int skl_calc_cdclk(int max_pixclk, int vco);
+ if (intel_crtc_has_dp_encoder(intel_crtc->config))
+ intel_dp_set_m_n(intel_crtc, M1_N1);
-static void intel_update_max_cdclk(struct drm_i915_private *dev_priv)
-{
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
- u32 limit = I915_READ(SKL_DFSM) & SKL_DFSM_CDCLK_LIMIT_MASK;
- int max_cdclk, vco;
+ intel_set_pipe_timings(intel_crtc);
+ intel_set_pipe_src_size(intel_crtc);
- vco = dev_priv->skl_preferred_vco_freq;
- WARN_ON(vco != 8100000 && vco != 8640000);
+ if (IS_CHERRYVIEW(dev_priv) && pipe == PIPE_B) {
+ struct drm_i915_private *dev_priv = to_i915(dev);
- /*
- * Use the lower (vco 8640) cdclk values as a
- * first guess. skl_calc_cdclk() will correct it
- * if the preferred vco is 8100 instead.
- */
- if (limit == SKL_DFSM_CDCLK_LIMIT_675)
- max_cdclk = 617143;
- else if (limit == SKL_DFSM_CDCLK_LIMIT_540)
- max_cdclk = 540000;
- else if (limit == SKL_DFSM_CDCLK_LIMIT_450)
- max_cdclk = 432000;
- else
- max_cdclk = 308571;
-
- dev_priv->max_cdclk_freq = skl_calc_cdclk(max_cdclk, vco);
- } else if (IS_GEMINILAKE(dev_priv)) {
- dev_priv->max_cdclk_freq = 316800;
- } else if (IS_BROXTON(dev_priv)) {
- dev_priv->max_cdclk_freq = 624000;
- } else if (IS_BROADWELL(dev_priv)) {
- /*
- * FIXME with extra cooling we can allow
- * 540 MHz for ULX and 675 Mhz for ULT.
- * How can we know if extra cooling is
- * available? PCI ID, VTB, something else?
- */
- if (I915_READ(FUSE_STRAP) & HSW_CDCLK_LIMIT)
- dev_priv->max_cdclk_freq = 450000;
- else if (IS_BDW_ULX(dev_priv))
- dev_priv->max_cdclk_freq = 450000;
- else if (IS_BDW_ULT(dev_priv))
- dev_priv->max_cdclk_freq = 540000;
- else
- dev_priv->max_cdclk_freq = 675000;
- } else if (IS_CHERRYVIEW(dev_priv)) {
- dev_priv->max_cdclk_freq = 320000;
- } else if (IS_VALLEYVIEW(dev_priv)) {
- dev_priv->max_cdclk_freq = 400000;
- } else {
- /* otherwise assume cdclk is fixed */
- dev_priv->max_cdclk_freq = dev_priv->cdclk_freq;
+ I915_WRITE(CHV_BLEND(pipe), CHV_BLEND_LEGACY);
+ I915_WRITE(CHV_CANVAS(pipe), 0);
}
- dev_priv->max_dotclk_freq = intel_compute_max_dotclk(dev_priv);
-
- DRM_DEBUG_DRIVER("Max CD clock rate: %d kHz\n",
- dev_priv->max_cdclk_freq);
+ i9xx_set_pipeconf(intel_crtc);
- DRM_DEBUG_DRIVER("Max dotclock rate: %d kHz\n",
- dev_priv->max_dotclk_freq);
-}
+ intel_crtc->active = true;
-static void intel_update_cdclk(struct drm_i915_private *dev_priv)
-{
- dev_priv->cdclk_freq = dev_priv->display.get_display_clock_speed(dev_priv);
+ intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
- if (INTEL_GEN(dev_priv) >= 9)
- DRM_DEBUG_DRIVER("Current CD clock rate: %d kHz, VCO: %d kHz, ref: %d kHz\n",
- dev_priv->cdclk_freq, dev_priv->cdclk_pll.vco,
- dev_priv->cdclk_pll.ref);
- else
- DRM_DEBUG_DRIVER("Current CD clock rate: %d kHz\n",
- dev_priv->cdclk_freq);
+ intel_encoders_pre_pll_enable(crtc, pipe_config, old_state);
- /*
- * 9:0 CMBUS [sic] CDCLK frequency (cdfreq):
- * Programmng [sic] note: bit[9:2] should be programmed to the number
- * of cdclk that generates 4MHz reference clock freq which is used to
- * generate GMBus clock. This will vary with the cdclk freq.
- */
- if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
- I915_WRITE(GMBUSFREQ_VLV, DIV_ROUND_UP(dev_priv->cdclk_freq, 1000));
-}
+ if (IS_CHERRYVIEW(dev_priv)) {
+ chv_prepare_pll(intel_crtc, intel_crtc->config);
+ chv_enable_pll(intel_crtc, intel_crtc->config);
+ } else {
+ vlv_prepare_pll(intel_crtc, intel_crtc->config);
+ vlv_enable_pll(intel_crtc, intel_crtc->config);
+ }
-/* convert from kHz to .1 fixpoint MHz with -1MHz offset */
-static int skl_cdclk_decimal(int cdclk)
-{
- return DIV_ROUND_CLOSEST(cdclk - 1000, 500);
-}
+ intel_encoders_pre_enable(crtc, pipe_config, old_state);
-static int bxt_de_pll_vco(struct drm_i915_private *dev_priv, int cdclk)
-{
- int ratio;
+ i9xx_pfit_enable(intel_crtc);
- if (cdclk == dev_priv->cdclk_pll.ref)
- return 0;
+ intel_color_load_luts(&pipe_config->base);
- switch (cdclk) {
- default:
- MISSING_CASE(cdclk);
- case 144000:
- case 288000:
- case 384000:
- case 576000:
- ratio = 60;
- break;
- case 624000:
- ratio = 65;
- break;
- }
-
- return dev_priv->cdclk_pll.ref * ratio;
-}
-
-static int glk_de_pll_vco(struct drm_i915_private *dev_priv, int cdclk)
-{
- int ratio;
-
- if (cdclk == dev_priv->cdclk_pll.ref)
- return 0;
-
- switch (cdclk) {
- default:
- MISSING_CASE(cdclk);
- case 79200:
- case 158400:
- case 316800:
- ratio = 33;
- break;
- }
-
- return dev_priv->cdclk_pll.ref * ratio;
-}
-
-static void bxt_de_pll_disable(struct drm_i915_private *dev_priv)
-{
- I915_WRITE(BXT_DE_PLL_ENABLE, 0);
-
- /* Timeout 200us */
- if (intel_wait_for_register(dev_priv,
- BXT_DE_PLL_ENABLE, BXT_DE_PLL_LOCK, 0,
- 1))
- DRM_ERROR("timeout waiting for DE PLL unlock\n");
-
- dev_priv->cdclk_pll.vco = 0;
-}
-
-static void bxt_de_pll_enable(struct drm_i915_private *dev_priv, int vco)
-{
- int ratio = DIV_ROUND_CLOSEST(vco, dev_priv->cdclk_pll.ref);
- u32 val;
-
- val = I915_READ(BXT_DE_PLL_CTL);
- val &= ~BXT_DE_PLL_RATIO_MASK;
- val |= BXT_DE_PLL_RATIO(ratio);
- I915_WRITE(BXT_DE_PLL_CTL, val);
-
- I915_WRITE(BXT_DE_PLL_ENABLE, BXT_DE_PLL_PLL_ENABLE);
-
- /* Timeout 200us */
- if (intel_wait_for_register(dev_priv,
- BXT_DE_PLL_ENABLE,
- BXT_DE_PLL_LOCK,
- BXT_DE_PLL_LOCK,
- 1))
- DRM_ERROR("timeout waiting for DE PLL lock\n");
-
- dev_priv->cdclk_pll.vco = vco;
-}
-
-static void bxt_set_cdclk(struct drm_i915_private *dev_priv, int cdclk)
-{
- u32 val, divider;
- int vco, ret;
-
- if (IS_GEMINILAKE(dev_priv))
- vco = glk_de_pll_vco(dev_priv, cdclk);
- else
- vco = bxt_de_pll_vco(dev_priv, cdclk);
-
- DRM_DEBUG_DRIVER("Changing CDCLK to %d kHz (VCO %d kHz)\n", cdclk, vco);
-
- /* cdclk = vco / 2 / div{1,1.5,2,4} */
- switch (DIV_ROUND_CLOSEST(vco, cdclk)) {
- case 8:
- divider = BXT_CDCLK_CD2X_DIV_SEL_4;
- break;
- case 4:
- divider = BXT_CDCLK_CD2X_DIV_SEL_2;
- break;
- case 3:
- WARN(IS_GEMINILAKE(dev_priv), "Unsupported divider\n");
- divider = BXT_CDCLK_CD2X_DIV_SEL_1_5;
- break;
- case 2:
- divider = BXT_CDCLK_CD2X_DIV_SEL_1;
- break;
- default:
- WARN_ON(cdclk != dev_priv->cdclk_pll.ref);
- WARN_ON(vco != 0);
-
- divider = BXT_CDCLK_CD2X_DIV_SEL_1;
- break;
- }
-
- /* Inform power controller of upcoming frequency change */
- mutex_lock(&dev_priv->rps.hw_lock);
- ret = sandybridge_pcode_write(dev_priv, HSW_PCODE_DE_WRITE_FREQ_REQ,
- 0x80000000);
- mutex_unlock(&dev_priv->rps.hw_lock);
-
- if (ret) {
- DRM_ERROR("PCode CDCLK freq change notify failed (err %d, freq %d)\n",
- ret, cdclk);
- return;
- }
-
- if (dev_priv->cdclk_pll.vco != 0 &&
- dev_priv->cdclk_pll.vco != vco)
- bxt_de_pll_disable(dev_priv);
-
- if (dev_priv->cdclk_pll.vco != vco)
- bxt_de_pll_enable(dev_priv, vco);
-
- val = divider | skl_cdclk_decimal(cdclk);
- /*
- * FIXME if only the cd2x divider needs changing, it could be done
- * without shutting off the pipe (if only one pipe is active).
- */
- val |= BXT_CDCLK_CD2X_PIPE_NONE;
- /*
- * Disable SSA Precharge when CD clock frequency < 500 MHz,
- * enable otherwise.
- */
- if (cdclk >= 500000)
- val |= BXT_CDCLK_SSA_PRECHARGE_ENABLE;
- I915_WRITE(CDCLK_CTL, val);
-
- mutex_lock(&dev_priv->rps.hw_lock);
- ret = sandybridge_pcode_write(dev_priv, HSW_PCODE_DE_WRITE_FREQ_REQ,
- DIV_ROUND_UP(cdclk, 25000));
- mutex_unlock(&dev_priv->rps.hw_lock);
-
- if (ret) {
- DRM_ERROR("PCode CDCLK freq set failed, (err %d, freq %d)\n",
- ret, cdclk);
- return;
- }
-
- intel_update_cdclk(dev_priv);
-}
-
-static void bxt_sanitize_cdclk(struct drm_i915_private *dev_priv)
-{
- u32 cdctl, expected;
-
- intel_update_cdclk(dev_priv);
-
- if (dev_priv->cdclk_pll.vco == 0 ||
- dev_priv->cdclk_freq == dev_priv->cdclk_pll.ref)
- goto sanitize;
-
- /* DPLL okay; verify the cdclock
- *
- * Some BIOS versions leave an incorrect decimal frequency value and
- * set reserved MBZ bits in CDCLK_CTL at least during exiting from S4,
- * so sanitize this register.
- */
- cdctl = I915_READ(CDCLK_CTL);
- /*
- * Let's ignore the pipe field, since BIOS could have configured the
- * dividers both synching to an active pipe, or asynchronously
- * (PIPE_NONE).
- */
- cdctl &= ~BXT_CDCLK_CD2X_PIPE_NONE;
-
- expected = (cdctl & BXT_CDCLK_CD2X_DIV_SEL_MASK) |
- skl_cdclk_decimal(dev_priv->cdclk_freq);
- /*
- * Disable SSA Precharge when CD clock frequency < 500 MHz,
- * enable otherwise.
- */
- if (dev_priv->cdclk_freq >= 500000)
- expected |= BXT_CDCLK_SSA_PRECHARGE_ENABLE;
-
- if (cdctl == expected)
- /* All well; nothing to sanitize */
- return;
-
-sanitize:
- DRM_DEBUG_KMS("Sanitizing cdclk programmed by pre-os\n");
-
- /* force cdclk programming */
- dev_priv->cdclk_freq = 0;
-
- /* force full PLL disable + enable */
- dev_priv->cdclk_pll.vco = -1;
-}
-
-void bxt_init_cdclk(struct drm_i915_private *dev_priv)
-{
- int cdclk;
-
- bxt_sanitize_cdclk(dev_priv);
-
- if (dev_priv->cdclk_freq != 0 && dev_priv->cdclk_pll.vco != 0)
- return;
-
- /*
- * FIXME:
- * - The initial CDCLK needs to be read from VBT.
- * Need to make this change after VBT has changes for BXT.
- */
- if (IS_GEMINILAKE(dev_priv))
- cdclk = glk_calc_cdclk(0);
- else
- cdclk = bxt_calc_cdclk(0);
-
- bxt_set_cdclk(dev_priv, cdclk);
-}
-
-void bxt_uninit_cdclk(struct drm_i915_private *dev_priv)
-{
- bxt_set_cdclk(dev_priv, dev_priv->cdclk_pll.ref);
-}
-
-static int skl_calc_cdclk(int max_pixclk, int vco)
-{
- if (vco == 8640000) {
- if (max_pixclk > 540000)
- return 617143;
- else if (max_pixclk > 432000)
- return 540000;
- else if (max_pixclk > 308571)
- return 432000;
- else
- return 308571;
- } else {
- if (max_pixclk > 540000)
- return 675000;
- else if (max_pixclk > 450000)
- return 540000;
- else if (max_pixclk > 337500)
- return 450000;
- else
- return 337500;
- }
-}
-
-static void
-skl_dpll0_update(struct drm_i915_private *dev_priv)
-{
- u32 val;
-
- dev_priv->cdclk_pll.ref = 24000;
- dev_priv->cdclk_pll.vco = 0;
-
- val = I915_READ(LCPLL1_CTL);
- if ((val & LCPLL_PLL_ENABLE) == 0)
- return;
-
- if (WARN_ON((val & LCPLL_PLL_LOCK) == 0))
- return;
-
- val = I915_READ(DPLL_CTRL1);
-
- if (WARN_ON((val & (DPLL_CTRL1_HDMI_MODE(SKL_DPLL0) |
- DPLL_CTRL1_SSC(SKL_DPLL0) |
- DPLL_CTRL1_OVERRIDE(SKL_DPLL0))) !=
- DPLL_CTRL1_OVERRIDE(SKL_DPLL0)))
- return;
-
- switch (val & DPLL_CTRL1_LINK_RATE_MASK(SKL_DPLL0)) {
- case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_810, SKL_DPLL0):
- case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1350, SKL_DPLL0):
- case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1620, SKL_DPLL0):
- case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_2700, SKL_DPLL0):
- dev_priv->cdclk_pll.vco = 8100000;
- break;
- case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1080, SKL_DPLL0):
- case DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_2160, SKL_DPLL0):
- dev_priv->cdclk_pll.vco = 8640000;
- break;
- default:
- MISSING_CASE(val & DPLL_CTRL1_LINK_RATE_MASK(SKL_DPLL0));
- break;
- }
-}
-
-void skl_set_preferred_cdclk_vco(struct drm_i915_private *dev_priv, int vco)
-{
- bool changed = dev_priv->skl_preferred_vco_freq != vco;
-
- dev_priv->skl_preferred_vco_freq = vco;
-
- if (changed)
- intel_update_max_cdclk(dev_priv);
-}
-
-static void
-skl_dpll0_enable(struct drm_i915_private *dev_priv, int vco)
-{
- int min_cdclk = skl_calc_cdclk(0, vco);
- u32 val;
-
- WARN_ON(vco != 8100000 && vco != 8640000);
-
- /* select the minimum CDCLK before enabling DPLL 0 */
- val = CDCLK_FREQ_337_308 | skl_cdclk_decimal(min_cdclk);
- I915_WRITE(CDCLK_CTL, val);
- POSTING_READ(CDCLK_CTL);
-
- /*
- * We always enable DPLL0 with the lowest link rate possible, but still
- * taking into account the VCO required to operate the eDP panel at the
- * desired frequency. The usual DP link rates operate with a VCO of
- * 8100 while the eDP 1.4 alternate link rates need a VCO of 8640.
- * The modeset code is responsible for the selection of the exact link
- * rate later on, with the constraint of choosing a frequency that
- * works with vco.
- */
- val = I915_READ(DPLL_CTRL1);
-
- val &= ~(DPLL_CTRL1_HDMI_MODE(SKL_DPLL0) | DPLL_CTRL1_SSC(SKL_DPLL0) |
- DPLL_CTRL1_LINK_RATE_MASK(SKL_DPLL0));
- val |= DPLL_CTRL1_OVERRIDE(SKL_DPLL0);
- if (vco == 8640000)
- val |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1080,
- SKL_DPLL0);
- else
- val |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_810,
- SKL_DPLL0);
-
- I915_WRITE(DPLL_CTRL1, val);
- POSTING_READ(DPLL_CTRL1);
-
- I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) | LCPLL_PLL_ENABLE);
-
- if (intel_wait_for_register(dev_priv,
- LCPLL1_CTL, LCPLL_PLL_LOCK, LCPLL_PLL_LOCK,
- 5))
- DRM_ERROR("DPLL0 not locked\n");
-
- dev_priv->cdclk_pll.vco = vco;
-
- /* We'll want to keep using the current vco from now on. */
- skl_set_preferred_cdclk_vco(dev_priv, vco);
-}
-
-static void
-skl_dpll0_disable(struct drm_i915_private *dev_priv)
-{
- I915_WRITE(LCPLL1_CTL, I915_READ(LCPLL1_CTL) & ~LCPLL_PLL_ENABLE);
- if (intel_wait_for_register(dev_priv,
- LCPLL1_CTL, LCPLL_PLL_LOCK, 0,
- 1))
- DRM_ERROR("Couldn't disable DPLL0\n");
-
- dev_priv->cdclk_pll.vco = 0;
-}
-
-static void skl_set_cdclk(struct drm_i915_private *dev_priv, int cdclk, int vco)
-{
- u32 freq_select, pcu_ack;
- int ret;
-
- WARN_ON((cdclk == 24000) != (vco == 0));
-
- DRM_DEBUG_DRIVER("Changing CDCLK to %d kHz (VCO %d kHz)\n", cdclk, vco);
-
- mutex_lock(&dev_priv->rps.hw_lock);
- ret = skl_pcode_request(dev_priv, SKL_PCODE_CDCLK_CONTROL,
- SKL_CDCLK_PREPARE_FOR_CHANGE,
- SKL_CDCLK_READY_FOR_CHANGE,
- SKL_CDCLK_READY_FOR_CHANGE, 3);
- mutex_unlock(&dev_priv->rps.hw_lock);
- if (ret) {
- DRM_ERROR("Failed to inform PCU about cdclk change (%d)\n",
- ret);
- return;
- }
-
- /* set CDCLK_CTL */
- switch (cdclk) {
- case 450000:
- case 432000:
- freq_select = CDCLK_FREQ_450_432;
- pcu_ack = 1;
- break;
- case 540000:
- freq_select = CDCLK_FREQ_540;
- pcu_ack = 2;
- break;
- case 308571:
- case 337500:
- default:
- freq_select = CDCLK_FREQ_337_308;
- pcu_ack = 0;
- break;
- case 617143:
- case 675000:
- freq_select = CDCLK_FREQ_675_617;
- pcu_ack = 3;
- break;
- }
-
- if (dev_priv->cdclk_pll.vco != 0 &&
- dev_priv->cdclk_pll.vco != vco)
- skl_dpll0_disable(dev_priv);
-
- if (dev_priv->cdclk_pll.vco != vco)
- skl_dpll0_enable(dev_priv, vco);
-
- I915_WRITE(CDCLK_CTL, freq_select | skl_cdclk_decimal(cdclk));
- POSTING_READ(CDCLK_CTL);
-
- /* inform PCU of the change */
- mutex_lock(&dev_priv->rps.hw_lock);
- sandybridge_pcode_write(dev_priv, SKL_PCODE_CDCLK_CONTROL, pcu_ack);
- mutex_unlock(&dev_priv->rps.hw_lock);
-
- intel_update_cdclk(dev_priv);
-}
-
-static void skl_sanitize_cdclk(struct drm_i915_private *dev_priv);
-
-void skl_uninit_cdclk(struct drm_i915_private *dev_priv)
-{
- skl_set_cdclk(dev_priv, dev_priv->cdclk_pll.ref, 0);
-}
-
-void skl_init_cdclk(struct drm_i915_private *dev_priv)
-{
- int cdclk, vco;
-
- skl_sanitize_cdclk(dev_priv);
-
- if (dev_priv->cdclk_freq != 0 && dev_priv->cdclk_pll.vco != 0) {
- /*
- * Use the current vco as our initial
- * guess as to what the preferred vco is.
- */
- if (dev_priv->skl_preferred_vco_freq == 0)
- skl_set_preferred_cdclk_vco(dev_priv,
- dev_priv->cdclk_pll.vco);
- return;
- }
-
- vco = dev_priv->skl_preferred_vco_freq;
- if (vco == 0)
- vco = 8100000;
- cdclk = skl_calc_cdclk(0, vco);
-
- skl_set_cdclk(dev_priv, cdclk, vco);
-}
-
-static void skl_sanitize_cdclk(struct drm_i915_private *dev_priv)
-{
- uint32_t cdctl, expected;
-
- /*
- * check if the pre-os intialized the display
- * There is SWF18 scratchpad register defined which is set by the
- * pre-os which can be used by the OS drivers to check the status
- */
- if ((I915_READ(SWF_ILK(0x18)) & 0x00FFFFFF) == 0)
- goto sanitize;
-
- intel_update_cdclk(dev_priv);
- /* Is PLL enabled and locked ? */
- if (dev_priv->cdclk_pll.vco == 0 ||
- dev_priv->cdclk_freq == dev_priv->cdclk_pll.ref)
- goto sanitize;
-
- /* DPLL okay; verify the cdclock
- *
- * Noticed in some instances that the freq selection is correct but
- * decimal part is programmed wrong from BIOS where pre-os does not
- * enable display. Verify the same as well.
- */
- cdctl = I915_READ(CDCLK_CTL);
- expected = (cdctl & CDCLK_FREQ_SEL_MASK) |
- skl_cdclk_decimal(dev_priv->cdclk_freq);
- if (cdctl == expected)
- /* All well; nothing to sanitize */
- return;
-
-sanitize:
- DRM_DEBUG_KMS("Sanitizing cdclk programmed by pre-os\n");
-
- /* force cdclk programming */
- dev_priv->cdclk_freq = 0;
- /* force full PLL disable + enable */
- dev_priv->cdclk_pll.vco = -1;
-}
-
-/* Adjust CDclk dividers to allow high res or save power if possible */
-static void valleyview_set_cdclk(struct drm_device *dev, int cdclk)
-{
- struct drm_i915_private *dev_priv = to_i915(dev);
- u32 val, cmd;
-
- WARN_ON(dev_priv->display.get_display_clock_speed(dev_priv)
- != dev_priv->cdclk_freq);
-
- if (cdclk >= 320000) /* jump to highest voltage for 400MHz too */
- cmd = 2;
- else if (cdclk == 266667)
- cmd = 1;
- else
- cmd = 0;
-
- mutex_lock(&dev_priv->rps.hw_lock);
- val = vlv_punit_read(dev_priv, PUNIT_REG_DSPFREQ);
- val &= ~DSPFREQGUAR_MASK;
- val |= (cmd << DSPFREQGUAR_SHIFT);
- vlv_punit_write(dev_priv, PUNIT_REG_DSPFREQ, val);
- if (wait_for((vlv_punit_read(dev_priv, PUNIT_REG_DSPFREQ) &
- DSPFREQSTAT_MASK) == (cmd << DSPFREQSTAT_SHIFT),
- 50)) {
- DRM_ERROR("timed out waiting for CDclk change\n");
- }
- mutex_unlock(&dev_priv->rps.hw_lock);
-
- mutex_lock(&dev_priv->sb_lock);
-
- if (cdclk == 400000) {
- u32 divider;
-
- divider = DIV_ROUND_CLOSEST(dev_priv->hpll_freq << 1, cdclk) - 1;
-
- /* adjust cdclk divider */
- val = vlv_cck_read(dev_priv, CCK_DISPLAY_CLOCK_CONTROL);
- val &= ~CCK_FREQUENCY_VALUES;
- val |= divider;
- vlv_cck_write(dev_priv, CCK_DISPLAY_CLOCK_CONTROL, val);
-
- if (wait_for((vlv_cck_read(dev_priv, CCK_DISPLAY_CLOCK_CONTROL) &
- CCK_FREQUENCY_STATUS) == (divider << CCK_FREQUENCY_STATUS_SHIFT),
- 50))
- DRM_ERROR("timed out waiting for CDclk change\n");
- }
-
- /* adjust self-refresh exit latency value */
- val = vlv_bunit_read(dev_priv, BUNIT_REG_BISOC);
- val &= ~0x7f;
-
- /*
- * For high bandwidth configs, we set a higher latency in the bunit
- * so that the core display fetch happens in time to avoid underruns.
- */
- if (cdclk == 400000)
- val |= 4500 / 250; /* 4.5 usec */
- else
- val |= 3000 / 250; /* 3.0 usec */
- vlv_bunit_write(dev_priv, BUNIT_REG_BISOC, val);
-
- mutex_unlock(&dev_priv->sb_lock);
-
- intel_update_cdclk(dev_priv);
-}
-
-static void cherryview_set_cdclk(struct drm_device *dev, int cdclk)
-{
- struct drm_i915_private *dev_priv = to_i915(dev);
- u32 val, cmd;
-
- WARN_ON(dev_priv->display.get_display_clock_speed(dev_priv)
- != dev_priv->cdclk_freq);
-
- switch (cdclk) {
- case 333333:
- case 320000:
- case 266667:
- case 200000:
- break;
- default:
- MISSING_CASE(cdclk);
- return;
- }
-
- /*
- * Specs are full of misinformation, but testing on actual
- * hardware has shown that we just need to write the desired
- * CCK divider into the Punit register.
- */
- cmd = DIV_ROUND_CLOSEST(dev_priv->hpll_freq << 1, cdclk) - 1;
-
- mutex_lock(&dev_priv->rps.hw_lock);
- val = vlv_punit_read(dev_priv, PUNIT_REG_DSPFREQ);
- val &= ~DSPFREQGUAR_MASK_CHV;
- val |= (cmd << DSPFREQGUAR_SHIFT_CHV);
- vlv_punit_write(dev_priv, PUNIT_REG_DSPFREQ, val);
- if (wait_for((vlv_punit_read(dev_priv, PUNIT_REG_DSPFREQ) &
- DSPFREQSTAT_MASK_CHV) == (cmd << DSPFREQSTAT_SHIFT_CHV),
- 50)) {
- DRM_ERROR("timed out waiting for CDclk change\n");
- }
- mutex_unlock(&dev_priv->rps.hw_lock);
-
- intel_update_cdclk(dev_priv);
-}
-
-static int valleyview_calc_cdclk(struct drm_i915_private *dev_priv,
- int max_pixclk)
-{
- int freq_320 = (dev_priv->hpll_freq << 1) % 320000 != 0 ? 333333 : 320000;
- int limit = IS_CHERRYVIEW(dev_priv) ? 95 : 90;
-
- /*
- * Really only a few cases to deal with, as only 4 CDclks are supported:
- * 200MHz
- * 267MHz
- * 320/333MHz (depends on HPLL freq)
- * 400MHz (VLV only)
- * So we check to see whether we're above 90% (VLV) or 95% (CHV)
- * of the lower bin and adjust if needed.
- *
- * We seem to get an unstable or solid color picture at 200MHz.
- * Not sure what's wrong. For now use 200MHz only when all pipes
- * are off.
- */
- if (!IS_CHERRYVIEW(dev_priv) &&
- max_pixclk > freq_320*limit/100)
- return 400000;
- else if (max_pixclk > 266667*limit/100)
- return freq_320;
- else if (max_pixclk > 0)
- return 266667;
- else
- return 200000;
-}
-
-static int glk_calc_cdclk(int max_pixclk)
-{
- if (max_pixclk > 2 * 158400)
- return 316800;
- else if (max_pixclk > 2 * 79200)
- return 158400;
- else
- return 79200;
-}
-
-static int bxt_calc_cdclk(int max_pixclk)
-{
- if (max_pixclk > 576000)
- return 624000;
- else if (max_pixclk > 384000)
- return 576000;
- else if (max_pixclk > 288000)
- return 384000;
- else if (max_pixclk > 144000)
- return 288000;
- else
- return 144000;
-}
-
-/* Compute the max pixel clock for new configuration. */
-static int intel_mode_max_pixclk(struct drm_device *dev,
- struct drm_atomic_state *state)
-{
- struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
- unsigned max_pixclk = 0, i;
- enum pipe pipe;
-
- memcpy(intel_state->min_pixclk, dev_priv->min_pixclk,
- sizeof(intel_state->min_pixclk));
-
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
- int pixclk = 0;
-
- if (crtc_state->enable)
- pixclk = crtc_state->adjusted_mode.crtc_clock;
-
- intel_state->min_pixclk[i] = pixclk;
- }
-
- for_each_pipe(dev_priv, pipe)
- max_pixclk = max(intel_state->min_pixclk[pipe], max_pixclk);
-
- return max_pixclk;
-}
-
-static int valleyview_modeset_calc_cdclk(struct drm_atomic_state *state)
-{
- struct drm_device *dev = state->dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- int max_pixclk = intel_mode_max_pixclk(dev, state);
- struct intel_atomic_state *intel_state =
- to_intel_atomic_state(state);
-
- intel_state->cdclk = intel_state->dev_cdclk =
- valleyview_calc_cdclk(dev_priv, max_pixclk);
-
- if (!intel_state->active_crtcs)
- intel_state->dev_cdclk = valleyview_calc_cdclk(dev_priv, 0);
-
- return 0;
-}
-
-static int bxt_modeset_calc_cdclk(struct drm_atomic_state *state)
-{
- struct drm_i915_private *dev_priv = to_i915(state->dev);
- int max_pixclk = ilk_max_pixel_rate(state);
- struct intel_atomic_state *intel_state =
- to_intel_atomic_state(state);
- int cdclk;
-
- if (IS_GEMINILAKE(dev_priv))
- cdclk = glk_calc_cdclk(max_pixclk);
- else
- cdclk = bxt_calc_cdclk(max_pixclk);
-
- intel_state->cdclk = intel_state->dev_cdclk = cdclk;
-
- if (!intel_state->active_crtcs) {
- if (IS_GEMINILAKE(dev_priv))
- cdclk = glk_calc_cdclk(0);
- else
- cdclk = bxt_calc_cdclk(0);
-
- intel_state->dev_cdclk = cdclk;
- }
-
- return 0;
-}
-
-static void vlv_program_pfi_credits(struct drm_i915_private *dev_priv)
-{
- unsigned int credits, default_credits;
-
- if (IS_CHERRYVIEW(dev_priv))
- default_credits = PFI_CREDIT(12);
- else
- default_credits = PFI_CREDIT(8);
-
- if (dev_priv->cdclk_freq >= dev_priv->czclk_freq) {
- /* CHV suggested value is 31 or 63 */
- if (IS_CHERRYVIEW(dev_priv))
- credits = PFI_CREDIT_63;
- else
- credits = PFI_CREDIT(15);
- } else {
- credits = default_credits;
- }
-
- /*
- * WA - write default credits before re-programming
- * FIXME: should we also set the resend bit here?
- */
- I915_WRITE(GCI_CONTROL, VGA_FAST_MODE_DISABLE |
- default_credits);
-
- I915_WRITE(GCI_CONTROL, VGA_FAST_MODE_DISABLE |
- credits | PFI_CREDIT_RESEND);
-
- /*
- * FIXME is this guaranteed to clear
- * immediately or should we poll for it?
- */
- WARN_ON(I915_READ(GCI_CONTROL) & PFI_CREDIT_RESEND);
-}
-
-static void valleyview_modeset_commit_cdclk(struct drm_atomic_state *old_state)
-{
- struct drm_device *dev = old_state->dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_atomic_state *old_intel_state =
- to_intel_atomic_state(old_state);
- unsigned req_cdclk = old_intel_state->dev_cdclk;
-
- /*
- * FIXME: We can end up here with all power domains off, yet
- * with a CDCLK frequency other than the minimum. To account
- * for this take the PIPE-A power domain, which covers the HW
- * blocks needed for the following programming. This can be
- * removed once it's guaranteed that we get here either with
- * the minimum CDCLK set, or the required power domains
- * enabled.
- */
- intel_display_power_get(dev_priv, POWER_DOMAIN_PIPE_A);
-
- if (IS_CHERRYVIEW(dev_priv))
- cherryview_set_cdclk(dev, req_cdclk);
- else
- valleyview_set_cdclk(dev, req_cdclk);
-
- vlv_program_pfi_credits(dev_priv);
-
- intel_display_power_put(dev_priv, POWER_DOMAIN_PIPE_A);
-}
-
-static void valleyview_crtc_enable(struct intel_crtc_state *pipe_config,
- struct drm_atomic_state *old_state)
-{
- struct drm_crtc *crtc = pipe_config->base.crtc;
- struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- int pipe = intel_crtc->pipe;
-
- if (WARN_ON(intel_crtc->active))
- return;
-
- if (intel_crtc_has_dp_encoder(intel_crtc->config))
- intel_dp_set_m_n(intel_crtc, M1_N1);
-
- intel_set_pipe_timings(intel_crtc);
- intel_set_pipe_src_size(intel_crtc);
-
- if (IS_CHERRYVIEW(dev_priv) && pipe == PIPE_B) {
- struct drm_i915_private *dev_priv = to_i915(dev);
-
- I915_WRITE(CHV_BLEND(pipe), CHV_BLEND_LEGACY);
- I915_WRITE(CHV_CANVAS(pipe), 0);
- }
-
- i9xx_set_pipeconf(intel_crtc);
-
- intel_crtc->active = true;
-
- intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
-
- intel_encoders_pre_pll_enable(crtc, pipe_config, old_state);
-
- if (IS_CHERRYVIEW(dev_priv)) {
- chv_prepare_pll(intel_crtc, intel_crtc->config);
- chv_enable_pll(intel_crtc, intel_crtc->config);
- } else {
- vlv_prepare_pll(intel_crtc, intel_crtc->config);
- vlv_enable_pll(intel_crtc, intel_crtc->config);
- }
-
- intel_encoders_pre_enable(crtc, pipe_config, old_state);
-
- i9xx_pfit_enable(intel_crtc);
-
- intel_color_load_luts(&pipe_config->base);
-
- intel_update_watermarks(intel_crtc);
+ dev_priv->display.initial_watermarks(old_intel_state,
+ pipe_config);
intel_enable_pipe(intel_crtc);
assert_vblank_disabled(crtc);
if (!IS_GEN2(dev_priv))
intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false);
+
+ if (!dev_priv->display.initial_watermarks)
+ intel_update_watermarks(intel_crtc);
}
static void intel_crtc_disable_noatomic(struct drm_crtc *crtc)
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct drm_i915_private *dev_priv = to_i915(crtc->dev);
enum intel_display_power_domain domain;
- unsigned long domains;
+ u64 domains;
struct drm_atomic_state *state;
struct intel_crtc_state *crtc_state;
int ret;
goto retry;
}
- if (needs_recompute)
- return RETRY;
-
- return ret;
-}
-
-static bool pipe_config_supports_ips(struct drm_i915_private *dev_priv,
- struct intel_crtc_state *pipe_config)
-{
- if (pipe_config->pipe_bpp > 24)
- return false;
-
- /* HSW can handle pixel rate up to cdclk? */
- if (IS_HASWELL(dev_priv))
- return true;
-
- /*
- * We compare against max which means we must take
- * the increased cdclk requirement into account when
- * calculating the new cdclk.
- *
- * Should measure whether using a lower cdclk w/o IPS
- */
- return ilk_pipe_pixel_rate(pipe_config) <=
- dev_priv->max_cdclk_freq * 95 / 100;
-}
-
-static void hsw_compute_ips_config(struct intel_crtc *crtc,
- struct intel_crtc_state *pipe_config)
-{
- struct drm_device *dev = crtc->base.dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
-
- pipe_config->ips_enabled = i915.enable_ips &&
- hsw_crtc_supports_ips(crtc) &&
- pipe_config_supports_ips(dev_priv, pipe_config);
-}
-
-static bool intel_crtc_supports_double_wide(const struct intel_crtc *crtc)
-{
- const struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
-
- /* GDG double wide on either pipe, otherwise pipe A only */
- return INTEL_INFO(dev_priv)->gen < 4 &&
- (crtc->pipe == PIPE_A || IS_I915G(dev_priv));
-}
-
-static int intel_crtc_compute_config(struct intel_crtc *crtc,
- struct intel_crtc_state *pipe_config)
-{
- struct drm_device *dev = crtc->base.dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- const struct drm_display_mode *adjusted_mode = &pipe_config->base.adjusted_mode;
- int clock_limit = dev_priv->max_dotclk_freq;
-
- if (INTEL_GEN(dev_priv) < 4) {
- clock_limit = dev_priv->max_cdclk_freq * 9 / 10;
-
- /*
- * Enable double wide mode when the dot clock
- * is > 90% of the (display) core speed.
- */
- if (intel_crtc_supports_double_wide(crtc) &&
- adjusted_mode->crtc_clock > clock_limit) {
- clock_limit = dev_priv->max_dotclk_freq;
- pipe_config->double_wide = true;
- }
- }
-
- if (adjusted_mode->crtc_clock > clock_limit) {
- DRM_DEBUG_KMS("requested pixel clock (%d kHz) too high (max: %d kHz, double wide: %s)\n",
- adjusted_mode->crtc_clock, clock_limit,
- yesno(pipe_config->double_wide));
- return -EINVAL;
- }
-
- /*
- * Pipe horizontal size must be even in:
- * - DVO ganged mode
- * - LVDS dual channel mode
- * - Double wide pipe
- */
- if ((intel_crtc_has_type(pipe_config, INTEL_OUTPUT_LVDS) &&
- intel_is_dual_link_lvds(dev)) || pipe_config->double_wide)
- pipe_config->pipe_src_w &= ~1;
-
- /* Cantiga+ cannot handle modes with a hsync front porch of 0.
- * WaPruneModeWithIncorrectHsyncOffset:ctg,elk,ilk,snb,ivb,vlv,hsw.
- */
- if ((INTEL_GEN(dev_priv) > 4 || IS_G4X(dev_priv)) &&
- adjusted_mode->crtc_hsync_start == adjusted_mode->crtc_hdisplay)
- return -EINVAL;
-
- if (HAS_IPS(dev_priv))
- hsw_compute_ips_config(crtc, pipe_config);
-
- if (pipe_config->has_pch_encoder)
- return ironlake_fdi_compute_config(crtc, pipe_config);
-
- return 0;
-}
-
-static int skylake_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- u32 cdctl;
-
- skl_dpll0_update(dev_priv);
-
- if (dev_priv->cdclk_pll.vco == 0)
- return dev_priv->cdclk_pll.ref;
-
- cdctl = I915_READ(CDCLK_CTL);
-
- if (dev_priv->cdclk_pll.vco == 8640000) {
- switch (cdctl & CDCLK_FREQ_SEL_MASK) {
- case CDCLK_FREQ_450_432:
- return 432000;
- case CDCLK_FREQ_337_308:
- return 308571;
- case CDCLK_FREQ_540:
- return 540000;
- case CDCLK_FREQ_675_617:
- return 617143;
- default:
- MISSING_CASE(cdctl & CDCLK_FREQ_SEL_MASK);
- }
- } else {
- switch (cdctl & CDCLK_FREQ_SEL_MASK) {
- case CDCLK_FREQ_450_432:
- return 450000;
- case CDCLK_FREQ_337_308:
- return 337500;
- case CDCLK_FREQ_540:
- return 540000;
- case CDCLK_FREQ_675_617:
- return 675000;
- default:
- MISSING_CASE(cdctl & CDCLK_FREQ_SEL_MASK);
- }
- }
-
- return dev_priv->cdclk_pll.ref;
-}
-
-static void bxt_de_pll_update(struct drm_i915_private *dev_priv)
-{
- u32 val;
-
- dev_priv->cdclk_pll.ref = 19200;
- dev_priv->cdclk_pll.vco = 0;
-
- val = I915_READ(BXT_DE_PLL_ENABLE);
- if ((val & BXT_DE_PLL_PLL_ENABLE) == 0)
- return;
-
- if (WARN_ON((val & BXT_DE_PLL_LOCK) == 0))
- return;
-
- val = I915_READ(BXT_DE_PLL_CTL);
- dev_priv->cdclk_pll.vco = (val & BXT_DE_PLL_RATIO_MASK) *
- dev_priv->cdclk_pll.ref;
-}
-
-static int broxton_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- u32 divider;
- int div, vco;
-
- bxt_de_pll_update(dev_priv);
-
- vco = dev_priv->cdclk_pll.vco;
- if (vco == 0)
- return dev_priv->cdclk_pll.ref;
-
- divider = I915_READ(CDCLK_CTL) & BXT_CDCLK_CD2X_DIV_SEL_MASK;
-
- switch (divider) {
- case BXT_CDCLK_CD2X_DIV_SEL_1:
- div = 2;
- break;
- case BXT_CDCLK_CD2X_DIV_SEL_1_5:
- WARN(IS_GEMINILAKE(dev_priv), "Unsupported divider\n");
- div = 3;
- break;
- case BXT_CDCLK_CD2X_DIV_SEL_2:
- div = 4;
- break;
- case BXT_CDCLK_CD2X_DIV_SEL_4:
- div = 8;
- break;
- default:
- MISSING_CASE(divider);
- return dev_priv->cdclk_pll.ref;
- }
-
- return DIV_ROUND_CLOSEST(vco, div);
-}
-
-static int broadwell_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- uint32_t lcpll = I915_READ(LCPLL_CTL);
- uint32_t freq = lcpll & LCPLL_CLK_FREQ_MASK;
-
- if (lcpll & LCPLL_CD_SOURCE_FCLK)
- return 800000;
- else if (I915_READ(FUSE_STRAP) & HSW_CDCLK_LIMIT)
- return 450000;
- else if (freq == LCPLL_CLK_FREQ_450)
- return 450000;
- else if (freq == LCPLL_CLK_FREQ_54O_BDW)
- return 540000;
- else if (freq == LCPLL_CLK_FREQ_337_5_BDW)
- return 337500;
- else
- return 675000;
-}
-
-static int haswell_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- uint32_t lcpll = I915_READ(LCPLL_CTL);
- uint32_t freq = lcpll & LCPLL_CLK_FREQ_MASK;
-
- if (lcpll & LCPLL_CD_SOURCE_FCLK)
- return 800000;
- else if (I915_READ(FUSE_STRAP) & HSW_CDCLK_LIMIT)
- return 450000;
- else if (freq == LCPLL_CLK_FREQ_450)
- return 450000;
- else if (IS_HSW_ULT(dev_priv))
- return 337500;
- else
- return 540000;
-}
-
-static int valleyview_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- return vlv_get_cck_clock_hpll(dev_priv, "cdclk",
- CCK_DISPLAY_CLOCK_CONTROL);
-}
-
-static int ilk_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- return 450000;
-}
-
-static int i945_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- return 400000;
-}
-
-static int i915_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- return 333333;
-}
-
-static int i9xx_misc_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- return 200000;
-}
-
-static int pnv_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- struct pci_dev *pdev = dev_priv->drm.pdev;
- u16 gcfgc = 0;
-
- pci_read_config_word(pdev, GCFGC, &gcfgc);
-
- switch (gcfgc & GC_DISPLAY_CLOCK_MASK) {
- case GC_DISPLAY_CLOCK_267_MHZ_PNV:
- return 266667;
- case GC_DISPLAY_CLOCK_333_MHZ_PNV:
- return 333333;
- case GC_DISPLAY_CLOCK_444_MHZ_PNV:
- return 444444;
- case GC_DISPLAY_CLOCK_200_MHZ_PNV:
- return 200000;
- default:
- DRM_ERROR("Unknown pnv display core clock 0x%04x\n", gcfgc);
- case GC_DISPLAY_CLOCK_133_MHZ_PNV:
- return 133333;
- case GC_DISPLAY_CLOCK_167_MHZ_PNV:
- return 166667;
- }
-}
-
-static int i915gm_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- struct pci_dev *pdev = dev_priv->drm.pdev;
- u16 gcfgc = 0;
-
- pci_read_config_word(pdev, GCFGC, &gcfgc);
-
- if (gcfgc & GC_LOW_FREQUENCY_ENABLE)
- return 133333;
- else {
- switch (gcfgc & GC_DISPLAY_CLOCK_MASK) {
- case GC_DISPLAY_CLOCK_333_MHZ:
- return 333333;
- default:
- case GC_DISPLAY_CLOCK_190_200_MHZ:
- return 190000;
- }
- }
-}
-
-static int i865_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- return 266667;
+ if (needs_recompute)
+ return RETRY;
+
+ return ret;
}
-static int i85x_get_display_clock_speed(struct drm_i915_private *dev_priv)
+static bool pipe_config_supports_ips(struct drm_i915_private *dev_priv,
+ struct intel_crtc_state *pipe_config)
{
- struct pci_dev *pdev = dev_priv->drm.pdev;
- u16 hpllcc = 0;
+ if (pipe_config->pipe_bpp > 24)
+ return false;
+
+ /* HSW can handle pixel rate up to cdclk? */
+ if (IS_HASWELL(dev_priv))
+ return true;
/*
- * 852GM/852GMV only supports 133 MHz and the HPLLCC
- * encoding is different :(
- * FIXME is this the right way to detect 852GM/852GMV?
+ * We compare against max which means we must take
+ * the increased cdclk requirement into account when
+ * calculating the new cdclk.
+ *
+ * Should measure whether using a lower cdclk w/o IPS
*/
- if (pdev->revision == 0x1)
- return 133333;
+ return pipe_config->pixel_rate <=
+ dev_priv->max_cdclk_freq * 95 / 100;
+}
- pci_bus_read_config_word(pdev->bus,
- PCI_DEVFN(0, 3), HPLLCC, &hpllcc);
+static void hsw_compute_ips_config(struct intel_crtc *crtc,
+ struct intel_crtc_state *pipe_config)
+{
+ struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
- /* Assume that the hardware is in the high speed state. This
- * should be the default.
- */
- switch (hpllcc & GC_CLOCK_CONTROL_MASK) {
- case GC_CLOCK_133_200:
- case GC_CLOCK_133_200_2:
- case GC_CLOCK_100_200:
- return 200000;
- case GC_CLOCK_166_250:
- return 250000;
- case GC_CLOCK_100_133:
- return 133333;
- case GC_CLOCK_133_266:
- case GC_CLOCK_133_266_2:
- case GC_CLOCK_166_266:
- return 266667;
- }
-
- /* Shouldn't happen */
- return 0;
+ pipe_config->ips_enabled = i915.enable_ips &&
+ hsw_crtc_supports_ips(crtc) &&
+ pipe_config_supports_ips(dev_priv, pipe_config);
}
-static int i830_get_display_clock_speed(struct drm_i915_private *dev_priv)
+static bool intel_crtc_supports_double_wide(const struct intel_crtc *crtc)
{
- return 133333;
+ const struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+
+ /* GDG double wide on either pipe, otherwise pipe A only */
+ return INTEL_INFO(dev_priv)->gen < 4 &&
+ (crtc->pipe == PIPE_A || IS_I915G(dev_priv));
}
-static unsigned int intel_hpll_vco(struct drm_i915_private *dev_priv)
+static uint32_t ilk_pipe_pixel_rate(const struct intel_crtc_state *pipe_config)
{
- static const unsigned int blb_vco[8] = {
- [0] = 3200000,
- [1] = 4000000,
- [2] = 5333333,
- [3] = 4800000,
- [4] = 6400000,
- };
- static const unsigned int pnv_vco[8] = {
- [0] = 3200000,
- [1] = 4000000,
- [2] = 5333333,
- [3] = 4800000,
- [4] = 2666667,
- };
- static const unsigned int cl_vco[8] = {
- [0] = 3200000,
- [1] = 4000000,
- [2] = 5333333,
- [3] = 6400000,
- [4] = 3333333,
- [5] = 3566667,
- [6] = 4266667,
- };
- static const unsigned int elk_vco[8] = {
- [0] = 3200000,
- [1] = 4000000,
- [2] = 5333333,
- [3] = 4800000,
- };
- static const unsigned int ctg_vco[8] = {
- [0] = 3200000,
- [1] = 4000000,
- [2] = 5333333,
- [3] = 6400000,
- [4] = 2666667,
- [5] = 4266667,
- };
- const unsigned int *vco_table;
- unsigned int vco;
- uint8_t tmp = 0;
-
- /* FIXME other chipsets? */
- if (IS_GM45(dev_priv))
- vco_table = ctg_vco;
- else if (IS_G4X(dev_priv))
- vco_table = elk_vco;
- else if (IS_I965GM(dev_priv))
- vco_table = cl_vco;
- else if (IS_PINEVIEW(dev_priv))
- vco_table = pnv_vco;
- else if (IS_G33(dev_priv))
- vco_table = blb_vco;
- else
- return 0;
+ uint32_t pixel_rate;
- tmp = I915_READ(IS_MOBILE(dev_priv) ? HPLLVCO_MOBILE : HPLLVCO);
+ pixel_rate = pipe_config->base.adjusted_mode.crtc_clock;
- vco = vco_table[tmp & 0x7];
- if (vco == 0)
- DRM_ERROR("Bad HPLL VCO (HPLLVCO=0x%02x)\n", tmp);
- else
- DRM_DEBUG_KMS("HPLL VCO %u kHz\n", vco);
+ /*
+ * We only use IF-ID interlacing. If we ever use
+ * PF-ID we'll need to adjust the pixel_rate here.
+ */
- return vco;
-}
+ if (pipe_config->pch_pfit.enabled) {
+ uint64_t pipe_w, pipe_h, pfit_w, pfit_h;
+ uint32_t pfit_size = pipe_config->pch_pfit.size;
-static int gm45_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- struct pci_dev *pdev = dev_priv->drm.pdev;
- unsigned int cdclk_sel, vco = intel_hpll_vco(dev_priv);
- uint16_t tmp = 0;
+ pipe_w = pipe_config->pipe_src_w;
+ pipe_h = pipe_config->pipe_src_h;
- pci_read_config_word(pdev, GCFGC, &tmp);
+ pfit_w = (pfit_size >> 16) & 0xFFFF;
+ pfit_h = pfit_size & 0xFFFF;
+ if (pipe_w < pfit_w)
+ pipe_w = pfit_w;
+ if (pipe_h < pfit_h)
+ pipe_h = pfit_h;
- cdclk_sel = (tmp >> 12) & 0x1;
+ if (WARN_ON(!pfit_w || !pfit_h))
+ return pixel_rate;
- switch (vco) {
- case 2666667:
- case 4000000:
- case 5333333:
- return cdclk_sel ? 333333 : 222222;
- case 3200000:
- return cdclk_sel ? 320000 : 228571;
- default:
- DRM_ERROR("Unable to determine CDCLK. HPLL VCO=%u, CFGC=0x%04x\n", vco, tmp);
- return 222222;
+ pixel_rate = div_u64((uint64_t) pixel_rate * pipe_w * pipe_h,
+ pfit_w * pfit_h);
}
+
+ return pixel_rate;
}
-static int i965gm_get_display_clock_speed(struct drm_i915_private *dev_priv)
+static void intel_crtc_compute_pixel_rate(struct intel_crtc_state *crtc_state)
{
- struct pci_dev *pdev = dev_priv->drm.pdev;
- static const uint8_t div_3200[] = { 16, 10, 8 };
- static const uint8_t div_4000[] = { 20, 12, 10 };
- static const uint8_t div_5333[] = { 24, 16, 14 };
- const uint8_t *div_table;
- unsigned int cdclk_sel, vco = intel_hpll_vco(dev_priv);
- uint16_t tmp = 0;
+ struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
- pci_read_config_word(pdev, GCFGC, &tmp);
+ if (HAS_GMCH_DISPLAY(dev_priv))
+ /* FIXME calculate proper pipe pixel rate for GMCH pfit */
+ crtc_state->pixel_rate =
+ crtc_state->base.adjusted_mode.crtc_clock;
+ else
+ crtc_state->pixel_rate =
+ ilk_pipe_pixel_rate(crtc_state);
+}
- cdclk_sel = ((tmp >> 8) & 0x1f) - 1;
+static int intel_crtc_compute_config(struct intel_crtc *crtc,
+ struct intel_crtc_state *pipe_config)
+{
+ struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ const struct drm_display_mode *adjusted_mode = &pipe_config->base.adjusted_mode;
+ int clock_limit = dev_priv->max_dotclk_freq;
- if (cdclk_sel >= ARRAY_SIZE(div_3200))
- goto fail;
+ if (INTEL_GEN(dev_priv) < 4) {
+ clock_limit = dev_priv->max_cdclk_freq * 9 / 10;
- switch (vco) {
- case 3200000:
- div_table = div_3200;
- break;
- case 4000000:
- div_table = div_4000;
- break;
- case 5333333:
- div_table = div_5333;
- break;
- default:
- goto fail;
+ /*
+ * Enable double wide mode when the dot clock
+ * is > 90% of the (display) core speed.
+ */
+ if (intel_crtc_supports_double_wide(crtc) &&
+ adjusted_mode->crtc_clock > clock_limit) {
+ clock_limit = dev_priv->max_dotclk_freq;
+ pipe_config->double_wide = true;
+ }
}
- return DIV_ROUND_CLOSEST(vco, div_table[cdclk_sel]);
-
-fail:
- DRM_ERROR("Unable to determine CDCLK. HPLL VCO=%u kHz, CFGC=0x%04x\n", vco, tmp);
- return 200000;
-}
-
-static int g33_get_display_clock_speed(struct drm_i915_private *dev_priv)
-{
- struct pci_dev *pdev = dev_priv->drm.pdev;
- static const uint8_t div_3200[] = { 12, 10, 8, 7, 5, 16 };
- static const uint8_t div_4000[] = { 14, 12, 10, 8, 6, 20 };
- static const uint8_t div_4800[] = { 20, 14, 12, 10, 8, 24 };
- static const uint8_t div_5333[] = { 20, 16, 12, 12, 8, 28 };
- const uint8_t *div_table;
- unsigned int cdclk_sel, vco = intel_hpll_vco(dev_priv);
- uint16_t tmp = 0;
+ if (adjusted_mode->crtc_clock > clock_limit) {
+ DRM_DEBUG_KMS("requested pixel clock (%d kHz) too high (max: %d kHz, double wide: %s)\n",
+ adjusted_mode->crtc_clock, clock_limit,
+ yesno(pipe_config->double_wide));
+ return -EINVAL;
+ }
- pci_read_config_word(pdev, GCFGC, &tmp);
+ /*
+ * Pipe horizontal size must be even in:
+ * - DVO ganged mode
+ * - LVDS dual channel mode
+ * - Double wide pipe
+ */
+ if ((intel_crtc_has_type(pipe_config, INTEL_OUTPUT_LVDS) &&
+ intel_is_dual_link_lvds(dev)) || pipe_config->double_wide)
+ pipe_config->pipe_src_w &= ~1;
- cdclk_sel = (tmp >> 4) & 0x7;
+ /* Cantiga+ cannot handle modes with a hsync front porch of 0.
+ * WaPruneModeWithIncorrectHsyncOffset:ctg,elk,ilk,snb,ivb,vlv,hsw.
+ */
+ if ((INTEL_GEN(dev_priv) > 4 || IS_G4X(dev_priv)) &&
+ adjusted_mode->crtc_hsync_start == adjusted_mode->crtc_hdisplay)
+ return -EINVAL;
- if (cdclk_sel >= ARRAY_SIZE(div_3200))
- goto fail;
+ intel_crtc_compute_pixel_rate(pipe_config);
- switch (vco) {
- case 3200000:
- div_table = div_3200;
- break;
- case 4000000:
- div_table = div_4000;
- break;
- case 4800000:
- div_table = div_4800;
- break;
- case 5333333:
- div_table = div_5333;
- break;
- default:
- goto fail;
- }
+ if (HAS_IPS(dev_priv))
+ hsw_compute_ips_config(crtc, pipe_config);
- return DIV_ROUND_CLOSEST(vco, div_table[cdclk_sel]);
+ if (pipe_config->has_pch_encoder)
+ return ironlake_fdi_compute_config(crtc, pipe_config);
-fail:
- DRM_ERROR("Unable to determine CDCLK. HPLL VCO=%u kHz, CFGC=0x%08x\n", vco, tmp);
- return 190476;
+ return 0;
}
static void
val = I915_READ(DSPSTRIDE(pipe));
fb->pitches[0] = val & 0xffffffc0;
- aligned_height = intel_fb_align_height(dev, fb->height,
- fb->format->format,
- fb->modifier);
+ aligned_height = intel_fb_align_height(fb, 0, fb->height);
plane_config->size = fb->pitches[0] * aligned_height;
fb->width = ((val >> 0) & 0x1fff) + 1;
val = I915_READ(PLANE_STRIDE(pipe, 0));
- stride_mult = intel_fb_stride_alignment(dev_priv, fb->modifier,
- fb->format->format);
+ stride_mult = intel_fb_stride_alignment(fb, 0);
fb->pitches[0] = (val & 0x3ff) * stride_mult;
- aligned_height = intel_fb_align_height(dev, fb->height,
- fb->format->format,
- fb->modifier);
+ aligned_height = intel_fb_align_height(fb, 0, fb->height);
plane_config->size = fb->pitches[0] * aligned_height;
val = I915_READ(DSPSTRIDE(pipe));
fb->pitches[0] = val & 0xffffffc0;
- aligned_height = intel_fb_align_height(dev, fb->height,
- fb->format->format,
- fb->modifier);
+ aligned_height = intel_fb_align_height(fb, 0, fb->height);
plane_config->size = fb->pitches[0] * aligned_height;
DRM_ERROR("LCPLL not locked yet\n");
if (val & LCPLL_CD_SOURCE_FCLK) {
- val = I915_READ(LCPLL_CTL);
- val &= ~LCPLL_CD_SOURCE_FCLK;
- I915_WRITE(LCPLL_CTL, val);
-
- if (wait_for_us((I915_READ(LCPLL_CTL) &
- LCPLL_CD_SOURCE_FCLK_DONE) == 0, 1))
- DRM_ERROR("Switching back to LCPLL failed\n");
- }
-
- intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
- intel_update_cdclk(dev_priv);
-}
-
-/*
- * Package states C8 and deeper are really deep PC states that can only be
- * reached when all the devices on the system allow it, so even if the graphics
- * device allows PC8+, it doesn't mean the system will actually get to these
- * states. Our driver only allows PC8+ when going into runtime PM.
- *
- * The requirements for PC8+ are that all the outputs are disabled, the power
- * well is disabled and most interrupts are disabled, and these are also
- * requirements for runtime PM. When these conditions are met, we manually do
- * the other conditions: disable the interrupts, clocks and switch LCPLL refclk
- * to Fclk. If we're in PC8+ and we get an non-hotplug interrupt, we can hard
- * hang the machine.
- *
- * When we really reach PC8 or deeper states (not just when we allow it) we lose
- * the state of some registers, so when we come back from PC8+ we need to
- * restore this state. We don't get into PC8+ if we're not in RC6, so we don't
- * need to take care of the registers kept by RC6. Notice that this happens even
- * if we don't put the device in PCI D3 state (which is what currently happens
- * because of the runtime PM support).
- *
- * For more, read "Display Sequences for Package C8" on the hardware
- * documentation.
- */
-void hsw_enable_pc8(struct drm_i915_private *dev_priv)
-{
- uint32_t val;
-
- DRM_DEBUG_KMS("Enabling package C8+\n");
-
- if (HAS_PCH_LPT_LP(dev_priv)) {
- val = I915_READ(SOUTH_DSPCLK_GATE_D);
- val &= ~PCH_LP_PARTITION_LEVEL_DISABLE;
- I915_WRITE(SOUTH_DSPCLK_GATE_D, val);
- }
-
- lpt_disable_clkout_dp(dev_priv);
- hsw_disable_lcpll(dev_priv, true, true);
-}
-
-void hsw_disable_pc8(struct drm_i915_private *dev_priv)
-{
- uint32_t val;
-
- DRM_DEBUG_KMS("Disabling package C8+\n");
-
- hsw_restore_lcpll(dev_priv);
- lpt_init_pch_refclk(dev_priv);
-
- if (HAS_PCH_LPT_LP(dev_priv)) {
- val = I915_READ(SOUTH_DSPCLK_GATE_D);
- val |= PCH_LP_PARTITION_LEVEL_DISABLE;
- I915_WRITE(SOUTH_DSPCLK_GATE_D, val);
- }
-}
-
-static void bxt_modeset_commit_cdclk(struct drm_atomic_state *old_state)
-{
- struct drm_device *dev = old_state->dev;
- struct intel_atomic_state *old_intel_state =
- to_intel_atomic_state(old_state);
- unsigned int req_cdclk = old_intel_state->dev_cdclk;
-
- bxt_set_cdclk(to_i915(dev), req_cdclk);
-}
-
-static int bdw_adjust_min_pipe_pixel_rate(struct intel_crtc_state *crtc_state,
- int pixel_rate)
-{
- struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
-
- /* pixel rate mustn't exceed 95% of cdclk with IPS on BDW */
- if (IS_BROADWELL(dev_priv) && crtc_state->ips_enabled)
- pixel_rate = DIV_ROUND_UP(pixel_rate * 100, 95);
-
- /* BSpec says "Do not use DisplayPort with CDCLK less than
- * 432 MHz, audio enabled, port width x4, and link rate
- * HBR2 (5.4 GHz), or else there may be audio corruption or
- * screen corruption."
- */
- if (intel_crtc_has_dp_encoder(crtc_state) &&
- crtc_state->has_audio &&
- crtc_state->port_clock >= 540000 &&
- crtc_state->lane_count == 4)
- pixel_rate = max(432000, pixel_rate);
-
- return pixel_rate;
-}
-
-/* compute the max rate for new configuration */
-static int ilk_max_pixel_rate(struct drm_atomic_state *state)
-{
- struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
- struct drm_i915_private *dev_priv = to_i915(state->dev);
- struct drm_crtc *crtc;
- struct drm_crtc_state *cstate;
- struct intel_crtc_state *crtc_state;
- unsigned max_pixel_rate = 0, i;
- enum pipe pipe;
-
- memcpy(intel_state->min_pixclk, dev_priv->min_pixclk,
- sizeof(intel_state->min_pixclk));
-
- for_each_crtc_in_state(state, crtc, cstate, i) {
- int pixel_rate;
-
- crtc_state = to_intel_crtc_state(cstate);
- if (!crtc_state->base.enable) {
- intel_state->min_pixclk[i] = 0;
- continue;
- }
-
- pixel_rate = ilk_pipe_pixel_rate(crtc_state);
-
- if (IS_BROADWELL(dev_priv) || IS_GEN9(dev_priv))
- pixel_rate = bdw_adjust_min_pipe_pixel_rate(crtc_state,
- pixel_rate);
-
- intel_state->min_pixclk[i] = pixel_rate;
- }
-
- for_each_pipe(dev_priv, pipe)
- max_pixel_rate = max(intel_state->min_pixclk[pipe], max_pixel_rate);
-
- return max_pixel_rate;
-}
-
-static void broadwell_set_cdclk(struct drm_device *dev, int cdclk)
-{
- struct drm_i915_private *dev_priv = to_i915(dev);
- uint32_t val, data;
- int ret;
-
- if (WARN((I915_READ(LCPLL_CTL) &
- (LCPLL_PLL_DISABLE | LCPLL_PLL_LOCK |
- LCPLL_CD_CLOCK_DISABLE | LCPLL_ROOT_CD_CLOCK_DISABLE |
- LCPLL_CD2X_CLOCK_DISABLE | LCPLL_POWER_DOWN_ALLOW |
- LCPLL_CD_SOURCE_FCLK)) != LCPLL_PLL_LOCK,
- "trying to change cdclk frequency with cdclk not enabled\n"))
- return;
-
- mutex_lock(&dev_priv->rps.hw_lock);
- ret = sandybridge_pcode_write(dev_priv,
- BDW_PCODE_DISPLAY_FREQ_CHANGE_REQ, 0x0);
- mutex_unlock(&dev_priv->rps.hw_lock);
- if (ret) {
- DRM_ERROR("failed to inform pcode about cdclk change\n");
- return;
- }
-
- val = I915_READ(LCPLL_CTL);
- val |= LCPLL_CD_SOURCE_FCLK;
- I915_WRITE(LCPLL_CTL, val);
-
- if (wait_for_us(I915_READ(LCPLL_CTL) &
- LCPLL_CD_SOURCE_FCLK_DONE, 1))
- DRM_ERROR("Switching to FCLK failed\n");
-
- val = I915_READ(LCPLL_CTL);
- val &= ~LCPLL_CLK_FREQ_MASK;
-
- switch (cdclk) {
- case 450000:
- val |= LCPLL_CLK_FREQ_450;
- data = 0;
- break;
- case 540000:
- val |= LCPLL_CLK_FREQ_54O_BDW;
- data = 1;
- break;
- case 337500:
- val |= LCPLL_CLK_FREQ_337_5_BDW;
- data = 2;
- break;
- case 675000:
- val |= LCPLL_CLK_FREQ_675_BDW;
- data = 3;
- break;
- default:
- WARN(1, "invalid cdclk frequency\n");
- return;
- }
-
- I915_WRITE(LCPLL_CTL, val);
-
- val = I915_READ(LCPLL_CTL);
- val &= ~LCPLL_CD_SOURCE_FCLK;
- I915_WRITE(LCPLL_CTL, val);
-
- if (wait_for_us((I915_READ(LCPLL_CTL) &
- LCPLL_CD_SOURCE_FCLK_DONE) == 0, 1))
- DRM_ERROR("Switching back to LCPLL failed\n");
-
- mutex_lock(&dev_priv->rps.hw_lock);
- sandybridge_pcode_write(dev_priv, HSW_PCODE_DE_WRITE_FREQ_REQ, data);
- mutex_unlock(&dev_priv->rps.hw_lock);
+ val = I915_READ(LCPLL_CTL);
+ val &= ~LCPLL_CD_SOURCE_FCLK;
+ I915_WRITE(LCPLL_CTL, val);
- I915_WRITE(CDCLK_FREQ, DIV_ROUND_CLOSEST(cdclk, 1000) - 1);
+ if (wait_for_us((I915_READ(LCPLL_CTL) &
+ LCPLL_CD_SOURCE_FCLK_DONE) == 0, 1))
+ DRM_ERROR("Switching back to LCPLL failed\n");
+ }
+ intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
intel_update_cdclk(dev_priv);
-
- WARN(cdclk != dev_priv->cdclk_freq,
- "cdclk requested %d kHz but got %d kHz\n",
- cdclk, dev_priv->cdclk_freq);
-}
-
-static int broadwell_calc_cdclk(int max_pixclk)
-{
- if (max_pixclk > 540000)
- return 675000;
- else if (max_pixclk > 450000)
- return 540000;
- else if (max_pixclk > 337500)
- return 450000;
- else
- return 337500;
}
-static int broadwell_modeset_calc_cdclk(struct drm_atomic_state *state)
+/*
+ * Package states C8 and deeper are really deep PC states that can only be
+ * reached when all the devices on the system allow it, so even if the graphics
+ * device allows PC8+, it doesn't mean the system will actually get to these
+ * states. Our driver only allows PC8+ when going into runtime PM.
+ *
+ * The requirements for PC8+ are that all the outputs are disabled, the power
+ * well is disabled and most interrupts are disabled, and these are also
+ * requirements for runtime PM. When these conditions are met, we manually do
+ * the other conditions: disable the interrupts, clocks and switch LCPLL refclk
+ * to Fclk. If we're in PC8+ and we get an non-hotplug interrupt, we can hard
+ * hang the machine.
+ *
+ * When we really reach PC8 or deeper states (not just when we allow it) we lose
+ * the state of some registers, so when we come back from PC8+ we need to
+ * restore this state. We don't get into PC8+ if we're not in RC6, so we don't
+ * need to take care of the registers kept by RC6. Notice that this happens even
+ * if we don't put the device in PCI D3 state (which is what currently happens
+ * because of the runtime PM support).
+ *
+ * For more, read "Display Sequences for Package C8" on the hardware
+ * documentation.
+ */
+void hsw_enable_pc8(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv = to_i915(state->dev);
- struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
- int max_pixclk = ilk_max_pixel_rate(state);
- int cdclk;
+ uint32_t val;
- /*
- * FIXME should also account for plane ratio
- * once 64bpp pixel formats are supported.
- */
- cdclk = broadwell_calc_cdclk(max_pixclk);
+ DRM_DEBUG_KMS("Enabling package C8+\n");
- if (cdclk > dev_priv->max_cdclk_freq) {
- DRM_DEBUG_KMS("requested cdclk (%d kHz) exceeds max (%d kHz)\n",
- cdclk, dev_priv->max_cdclk_freq);
- return -EINVAL;
+ if (HAS_PCH_LPT_LP(dev_priv)) {
+ val = I915_READ(SOUTH_DSPCLK_GATE_D);
+ val &= ~PCH_LP_PARTITION_LEVEL_DISABLE;
+ I915_WRITE(SOUTH_DSPCLK_GATE_D, val);
}
- intel_state->cdclk = intel_state->dev_cdclk = cdclk;
- if (!intel_state->active_crtcs)
- intel_state->dev_cdclk = broadwell_calc_cdclk(0);
-
- return 0;
+ lpt_disable_clkout_dp(dev_priv);
+ hsw_disable_lcpll(dev_priv, true, true);
}
-static void broadwell_modeset_commit_cdclk(struct drm_atomic_state *old_state)
+void hsw_disable_pc8(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = old_state->dev;
- struct intel_atomic_state *old_intel_state =
- to_intel_atomic_state(old_state);
- unsigned req_cdclk = old_intel_state->dev_cdclk;
-
- broadwell_set_cdclk(dev, req_cdclk);
-}
+ uint32_t val;
-static int skl_modeset_calc_cdclk(struct drm_atomic_state *state)
-{
- struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
- struct drm_i915_private *dev_priv = to_i915(state->dev);
- const int max_pixclk = ilk_max_pixel_rate(state);
- int vco = intel_state->cdclk_pll_vco;
- int cdclk;
+ DRM_DEBUG_KMS("Disabling package C8+\n");
- /*
- * FIXME should also account for plane ratio
- * once 64bpp pixel formats are supported.
- */
- cdclk = skl_calc_cdclk(max_pixclk, vco);
+ hsw_restore_lcpll(dev_priv);
+ lpt_init_pch_refclk(dev_priv);
- /*
- * FIXME move the cdclk caclulation to
- * compute_config() so we can fail gracegully.
- */
- if (cdclk > dev_priv->max_cdclk_freq) {
- DRM_ERROR("requested cdclk (%d kHz) exceeds max (%d kHz)\n",
- cdclk, dev_priv->max_cdclk_freq);
- cdclk = dev_priv->max_cdclk_freq;
+ if (HAS_PCH_LPT_LP(dev_priv)) {
+ val = I915_READ(SOUTH_DSPCLK_GATE_D);
+ val |= PCH_LP_PARTITION_LEVEL_DISABLE;
+ I915_WRITE(SOUTH_DSPCLK_GATE_D, val);
}
-
- intel_state->cdclk = intel_state->dev_cdclk = cdclk;
- if (!intel_state->active_crtcs)
- intel_state->dev_cdclk = skl_calc_cdclk(0, vco);
-
- return 0;
-}
-
-static void skl_modeset_commit_cdclk(struct drm_atomic_state *old_state)
-{
- struct drm_i915_private *dev_priv = to_i915(old_state->dev);
- struct intel_atomic_state *intel_state = to_intel_atomic_state(old_state);
- unsigned int req_cdclk = intel_state->dev_cdclk;
- unsigned int req_vco = intel_state->cdclk_pll_vco;
-
- skl_set_cdclk(dev_priv, req_cdclk, req_vco);
}
static int haswell_crtc_compute_clock(struct intel_crtc *crtc,
static bool hsw_get_transcoder_state(struct intel_crtc *crtc,
struct intel_crtc_state *pipe_config,
- unsigned long *power_domain_mask)
+ u64 *power_domain_mask)
{
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
power_domain = POWER_DOMAIN_TRANSCODER(pipe_config->cpu_transcoder);
if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
return false;
- *power_domain_mask |= BIT(power_domain);
+ *power_domain_mask |= BIT_ULL(power_domain);
tmp = I915_READ(PIPECONF(pipe_config->cpu_transcoder));
static bool bxt_get_dsi_transcoder_state(struct intel_crtc *crtc,
struct intel_crtc_state *pipe_config,
- unsigned long *power_domain_mask)
+ u64 *power_domain_mask)
{
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
power_domain = POWER_DOMAIN_TRANSCODER(cpu_transcoder);
if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
continue;
- *power_domain_mask |= BIT(power_domain);
+ *power_domain_mask |= BIT_ULL(power_domain);
/*
* The PLL needs to be enabled with a valid divider
port = (tmp & TRANS_DDI_PORT_MASK) >> TRANS_DDI_PORT_SHIFT;
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ if (IS_GEN9_BC(dev_priv))
skylake_get_ddi_pll(dev_priv, port, pipe_config);
else if (IS_GEN9_LP(dev_priv))
bxt_get_ddi_pll(dev_priv, port, pipe_config);
{
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
enum intel_display_power_domain power_domain;
- unsigned long power_domain_mask;
+ u64 power_domain_mask;
bool active;
power_domain = POWER_DOMAIN_PIPE(crtc->pipe);
if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
return false;
- power_domain_mask = BIT(power_domain);
+ power_domain_mask = BIT_ULL(power_domain);
pipe_config->shared_dpll = NULL;
power_domain = POWER_DOMAIN_PIPE_PANEL_FITTER(crtc->pipe);
if (intel_display_power_get_if_enabled(dev_priv, power_domain)) {
- power_domain_mask |= BIT(power_domain);
+ power_domain_mask |= BIT_ULL(power_domain);
if (INTEL_GEN(dev_priv) >= 9)
skylake_get_pfit_config(crtc, pipe_config);
else
/* On these chipsets we can only modify the base/size/stride
* whilst the cursor is disabled.
*/
- I915_WRITE(CURCNTR(PIPE_A), 0);
- POSTING_READ(CURCNTR(PIPE_A));
+ I915_WRITE_FW(CURCNTR(PIPE_A), 0);
+ POSTING_READ_FW(CURCNTR(PIPE_A));
intel_crtc->cursor_cntl = 0;
}
if (intel_crtc->cursor_base != base) {
- I915_WRITE(CURBASE(PIPE_A), base);
+ I915_WRITE_FW(CURBASE(PIPE_A), base);
intel_crtc->cursor_base = base;
}
if (intel_crtc->cursor_size != size) {
- I915_WRITE(CURSIZE, size);
+ I915_WRITE_FW(CURSIZE, size);
intel_crtc->cursor_size = size;
}
if (intel_crtc->cursor_cntl != cntl) {
- I915_WRITE(CURCNTR(PIPE_A), cntl);
- POSTING_READ(CURCNTR(PIPE_A));
+ I915_WRITE_FW(CURCNTR(PIPE_A), cntl);
+ POSTING_READ_FW(CURCNTR(PIPE_A));
intel_crtc->cursor_cntl = cntl;
}
}
}
if (intel_crtc->cursor_cntl != cntl) {
- I915_WRITE(CURCNTR(pipe), cntl);
- POSTING_READ(CURCNTR(pipe));
+ I915_WRITE_FW(CURCNTR(pipe), cntl);
+ POSTING_READ_FW(CURCNTR(pipe));
intel_crtc->cursor_cntl = cntl;
}
/* and commit changes on next vblank */
- I915_WRITE(CURBASE(pipe), base);
- POSTING_READ(CURBASE(pipe));
+ I915_WRITE_FW(CURBASE(pipe), base);
+ POSTING_READ_FW(CURBASE(pipe));
intel_crtc->cursor_base = base;
}
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
int pipe = intel_crtc->pipe;
u32 base = intel_crtc->cursor_addr;
+ unsigned long irqflags;
u32 pos = 0;
if (plane_state) {
}
}
- I915_WRITE(CURPOS(pipe), pos);
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
+ I915_WRITE_FW(CURPOS(pipe), pos);
if (IS_I845G(dev_priv) || IS_I865G(dev_priv))
i845_update_cursor(crtc, base, plane_state);
else
i9xx_update_cursor(crtc, base, plane_state);
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static bool cursor_size_ok(struct drm_i915_private *dev_priv,
};
struct drm_framebuffer *
-__intel_framebuffer_create(struct drm_device *dev,
- struct drm_mode_fb_cmd2 *mode_cmd,
- struct drm_i915_gem_object *obj)
+intel_framebuffer_create(struct drm_i915_gem_object *obj,
+ struct drm_mode_fb_cmd2 *mode_cmd)
{
struct intel_framebuffer *intel_fb;
int ret;
if (!intel_fb)
return ERR_PTR(-ENOMEM);
- ret = intel_framebuffer_init(dev, intel_fb, mode_cmd, obj);
+ ret = intel_framebuffer_init(intel_fb, obj, mode_cmd);
if (ret)
goto err;
return ERR_PTR(ret);
}
-static struct drm_framebuffer *
-intel_framebuffer_create(struct drm_device *dev,
- struct drm_mode_fb_cmd2 *mode_cmd,
- struct drm_i915_gem_object *obj)
-{
- struct drm_framebuffer *fb;
- int ret;
-
- ret = i915_mutex_lock_interruptible(dev);
- if (ret)
- return ERR_PTR(ret);
- fb = __intel_framebuffer_create(dev, mode_cmd, obj);
- mutex_unlock(&dev->struct_mutex);
-
- return fb;
-}
-
static u32
intel_framebuffer_pitch_for_width(int width, int bpp)
{
bpp);
mode_cmd.pixel_format = drm_mode_legacy_fb_format(bpp, depth);
- fb = intel_framebuffer_create(dev, &mode_cmd, obj);
+ fb = intel_framebuffer_create(obj, &mode_cmd);
if (IS_ERR(fb))
i915_gem_object_put(obj);
if (!state)
return;
- ret = drm_atomic_commit(state);
+ ret = drm_atomic_helper_commit_duplicated_state(state, ctx);
if (ret)
DRM_DEBUG_KMS("Couldn't release load detect pipe: %i\n", ret);
drm_atomic_state_put(state);
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ring *ring = req->ring;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- u32 flip_mask;
- int ret;
+ u32 flip_mask, *cs;
- ret = intel_ring_begin(req, 6);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 6);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
/* Can't queue multiple flips, so wait for the previous
* one to finish before executing the next.
flip_mask = MI_WAIT_FOR_PLANE_B_FLIP;
else
flip_mask = MI_WAIT_FOR_PLANE_A_FLIP;
- intel_ring_emit(ring, MI_WAIT_FOR_EVENT | flip_mask);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_emit(ring, MI_DISPLAY_FLIP |
- MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(ring, fb->pitches[0]);
- intel_ring_emit(ring, intel_crtc->flip_work->gtt_offset);
- intel_ring_emit(ring, 0); /* aux display base address, unused */
+ *cs++ = MI_WAIT_FOR_EVENT | flip_mask;
+ *cs++ = MI_NOOP;
+ *cs++ = MI_DISPLAY_FLIP | MI_DISPLAY_FLIP_PLANE(intel_crtc->plane);
+ *cs++ = fb->pitches[0];
+ *cs++ = intel_crtc->flip_work->gtt_offset;
+ *cs++ = 0; /* aux display base address, unused */
return 0;
}
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ring *ring = req->ring;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- u32 flip_mask;
- int ret;
+ u32 flip_mask, *cs;
- ret = intel_ring_begin(req, 6);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 6);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
if (intel_crtc->plane)
flip_mask = MI_WAIT_FOR_PLANE_B_FLIP;
else
flip_mask = MI_WAIT_FOR_PLANE_A_FLIP;
- intel_ring_emit(ring, MI_WAIT_FOR_EVENT | flip_mask);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_emit(ring, MI_DISPLAY_FLIP_I915 |
- MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(ring, fb->pitches[0]);
- intel_ring_emit(ring, intel_crtc->flip_work->gtt_offset);
- intel_ring_emit(ring, MI_NOOP);
+ *cs++ = MI_WAIT_FOR_EVENT | flip_mask;
+ *cs++ = MI_NOOP;
+ *cs++ = MI_DISPLAY_FLIP_I915 | MI_DISPLAY_FLIP_PLANE(intel_crtc->plane);
+ *cs++ = fb->pitches[0];
+ *cs++ = intel_crtc->flip_work->gtt_offset;
+ *cs++ = MI_NOOP;
return 0;
}
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ring *ring = req->ring;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- uint32_t pf, pipesrc;
- int ret;
+ u32 pf, pipesrc, *cs;
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
/* i965+ uses the linear or tiled offsets from the
* Display Registers (which do not change across a page-flip)
* so we need only reprogram the base address.
*/
- intel_ring_emit(ring, MI_DISPLAY_FLIP |
- MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(ring, fb->pitches[0]);
- intel_ring_emit(ring, intel_crtc->flip_work->gtt_offset |
- intel_fb_modifier_to_tiling(fb->modifier));
+ *cs++ = MI_DISPLAY_FLIP | MI_DISPLAY_FLIP_PLANE(intel_crtc->plane);
+ *cs++ = fb->pitches[0];
+ *cs++ = intel_crtc->flip_work->gtt_offset |
+ intel_fb_modifier_to_tiling(fb->modifier);
/* XXX Enabling the panel-fitter across page-flip is so far
* untested on non-native modes, so ignore it for now.
*/
pf = 0;
pipesrc = I915_READ(PIPESRC(intel_crtc->pipe)) & 0x0fff0fff;
- intel_ring_emit(ring, pf | pipesrc);
+ *cs++ = pf | pipesrc;
return 0;
}
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ring *ring = req->ring;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- uint32_t pf, pipesrc;
- int ret;
+ u32 pf, pipesrc, *cs;
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, MI_DISPLAY_FLIP |
- MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(ring, fb->pitches[0] |
- intel_fb_modifier_to_tiling(fb->modifier));
- intel_ring_emit(ring, intel_crtc->flip_work->gtt_offset);
+ *cs++ = MI_DISPLAY_FLIP | MI_DISPLAY_FLIP_PLANE(intel_crtc->plane);
+ *cs++ = fb->pitches[0] | intel_fb_modifier_to_tiling(fb->modifier);
+ *cs++ = intel_crtc->flip_work->gtt_offset;
/* Contrary to the suggestions in the documentation,
* "Enable Panel Fitter" does not seem to be required when page
*/
pf = 0;
pipesrc = I915_READ(PIPESRC(intel_crtc->pipe)) & 0x0fff0fff;
- intel_ring_emit(ring, pf | pipesrc);
+ *cs++ = pf | pipesrc;
return 0;
}
uint32_t flags)
{
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_ring *ring = req->ring;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- uint32_t plane_bit = 0;
+ u32 *cs, plane_bit = 0;
int len, ret;
switch (intel_crtc->plane) {
if (ret)
return ret;
- ret = intel_ring_begin(req, len);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, len);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
/* Unmask the flip-done completion message. Note that the bspec says that
* we should do this for both the BCS and RCS, and that we must not unmask
* to zero does lead to lockups within MI_DISPLAY_FLIP.
*/
if (req->engine->id == RCS) {
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ring, DERRMR);
- intel_ring_emit(ring, ~(DERRMR_PIPEA_PRI_FLIP_DONE |
- DERRMR_PIPEB_PRI_FLIP_DONE |
- DERRMR_PIPEC_PRI_FLIP_DONE));
+ *cs++ = MI_LOAD_REGISTER_IMM(1);
+ *cs++ = i915_mmio_reg_offset(DERRMR);
+ *cs++ = ~(DERRMR_PIPEA_PRI_FLIP_DONE |
+ DERRMR_PIPEB_PRI_FLIP_DONE |
+ DERRMR_PIPEC_PRI_FLIP_DONE);
if (IS_GEN8(dev_priv))
- intel_ring_emit(ring, MI_STORE_REGISTER_MEM_GEN8 |
- MI_SRM_LRM_GLOBAL_GTT);
+ *cs++ = MI_STORE_REGISTER_MEM_GEN8 |
+ MI_SRM_LRM_GLOBAL_GTT;
else
- intel_ring_emit(ring, MI_STORE_REGISTER_MEM |
- MI_SRM_LRM_GLOBAL_GTT);
- intel_ring_emit_reg(ring, DERRMR);
- intel_ring_emit(ring,
- i915_ggtt_offset(req->engine->scratch) + 256);
+ *cs++ = MI_STORE_REGISTER_MEM | MI_SRM_LRM_GLOBAL_GTT;
+ *cs++ = i915_mmio_reg_offset(DERRMR);
+ *cs++ = i915_ggtt_offset(req->engine->scratch) + 256;
if (IS_GEN8(dev_priv)) {
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, MI_NOOP);
+ *cs++ = 0;
+ *cs++ = MI_NOOP;
}
}
- intel_ring_emit(ring, MI_DISPLAY_FLIP_I915 | plane_bit);
- intel_ring_emit(ring, fb->pitches[0] |
- intel_fb_modifier_to_tiling(fb->modifier));
- intel_ring_emit(ring, intel_crtc->flip_work->gtt_offset);
- intel_ring_emit(ring, (MI_NOOP));
+ *cs++ = MI_DISPLAY_FLIP_I915 | plane_bit;
+ *cs++ = fb->pitches[0] | intel_fb_modifier_to_tiling(fb->modifier);
+ *cs++ = intel_crtc->flip_work->gtt_offset;
+ *cs++ = MI_NOOP;
return 0;
}
spin_unlock(&dev->event_lock);
}
+__maybe_unused
static int intel_crtc_page_flip(struct drm_crtc *crtc,
struct drm_framebuffer *fb,
struct drm_pending_vblank_event *event,
goto cleanup;
intel_crtc->reset_count = i915_reset_count(&dev_priv->gpu_error);
- if (i915_reset_in_progress_or_wedged(&dev_priv->gpu_error)) {
+ if (i915_reset_backoff_or_wedged(&dev_priv->gpu_error)) {
ret = -EIO;
goto unlock;
}
intel_mark_page_flip_active(intel_crtc, work);
work->flip_queued_req = i915_gem_request_get(request);
- i915_add_request_no_flush(request);
+ i915_add_request(request);
}
i915_gem_object_wait_priority(obj, 0, I915_PRIORITY_DISPLAY);
return 0;
cleanup_request:
- i915_add_request_no_flush(request);
+ i915_add_request(request);
cleanup_unpin:
to_intel_plane_state(primary->state)->vma = work->old_vma;
intel_unpin_fb_vma(vma);
struct intel_crtc_state *pipe_config = to_intel_crtc_state(crtc_state);
struct drm_crtc *crtc = crtc_state->crtc;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- struct drm_plane *plane = plane_state->plane;
+ struct intel_plane *plane = to_intel_plane(plane_state->plane);
struct drm_device *dev = crtc->dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_plane_state *old_plane_state =
- to_intel_plane_state(plane->state);
+ to_intel_plane_state(plane->base.state);
bool mode_changed = needs_modeset(crtc_state);
bool was_crtc_enabled = crtc->state->active;
bool is_crtc_enabled = crtc_state->active;
struct drm_framebuffer *fb = plane_state->fb;
int ret;
- if (INTEL_GEN(dev_priv) >= 9 && plane->type != DRM_PLANE_TYPE_CURSOR) {
+ if (INTEL_GEN(dev_priv) >= 9 && plane->id != PLANE_CURSOR) {
ret = skl_update_scaler_plane(
to_intel_crtc_state(crtc_state),
to_intel_plane_state(plane_state));
* per-plane wm computation to the .check_plane() hook, and
* only combine the results from all planes in the current place?
*/
- if (!is_crtc_enabled)
+ if (!is_crtc_enabled) {
plane_state->visible = visible = false;
+ to_intel_crtc_state(crtc_state)->active_planes &= ~BIT(plane->id);
+ }
if (!was_visible && !visible)
return 0;
turn_on = visible && (!was_visible || mode_changed);
DRM_DEBUG_ATOMIC("[CRTC:%d:%s] has [PLANE:%d:%s] with fb %i\n",
- intel_crtc->base.base.id,
- intel_crtc->base.name,
- plane->base.id, plane->name,
+ intel_crtc->base.base.id, intel_crtc->base.name,
+ plane->base.base.id, plane->base.name,
fb ? fb->base.id : -1);
DRM_DEBUG_ATOMIC("[PLANE:%d:%s] visible %i -> %i, off %i, on %i, ms %i\n",
- plane->base.id, plane->name,
+ plane->base.base.id, plane->base.name,
was_visible, visible,
turn_off, turn_on, mode_changed);
if (turn_on) {
- pipe_config->update_wm_pre = true;
+ if (INTEL_GEN(dev_priv) < 5)
+ pipe_config->update_wm_pre = true;
/* must disable cxsr around plane enable/disable */
- if (plane->type != DRM_PLANE_TYPE_CURSOR)
+ if (plane->id != PLANE_CURSOR)
pipe_config->disable_cxsr = true;
} else if (turn_off) {
- pipe_config->update_wm_post = true;
+ if (INTEL_GEN(dev_priv) < 5)
+ pipe_config->update_wm_post = true;
/* must disable cxsr around plane enable/disable */
- if (plane->type != DRM_PLANE_TYPE_CURSOR)
+ if (plane->id != PLANE_CURSOR)
pipe_config->disable_cxsr = true;
- } else if (intel_wm_need_update(plane, plane_state)) {
- /* FIXME bollocks */
- pipe_config->update_wm_pre = true;
- pipe_config->update_wm_post = true;
+ } else if (intel_wm_need_update(&plane->base, plane_state)) {
+ if (INTEL_GEN(dev_priv) < 5) {
+ /* FIXME bollocks */
+ pipe_config->update_wm_pre = true;
+ pipe_config->update_wm_post = true;
+ }
}
- /* Pre-gen9 platforms need two-step watermark updates */
- if ((pipe_config->update_wm_pre || pipe_config->update_wm_post) &&
- INTEL_GEN(dev_priv) < 9 && dev_priv->display.optimize_watermarks)
- to_intel_crtc_state(crtc_state)->wm.need_postvbl_update = true;
-
if (visible || was_visible)
- pipe_config->fb_bits |= to_intel_plane(plane)->frontbuffer_bit;
+ pipe_config->fb_bits |= plane->frontbuffer_bit;
/*
* WaCxSRDisabledForSpriteScaling:ivb
* cstate->update_wm was already set above, so this flag will
* take effect when we commit and program watermarks.
*/
- if (plane->type == DRM_PLANE_TYPE_OVERLAY && IS_IVYBRIDGE(dev_priv) &&
+ if (plane->id == PLANE_SPRITE0 && IS_IVYBRIDGE(dev_priv) &&
needs_scaling(to_intel_plane_state(plane_state)) &&
!needs_scaling(old_plane_state))
pipe_config->disable_lp_wm = true;
struct drm_connector_state *connector_state;
int i;
- for_each_connector_in_state(state, connector, connector_state, i) {
+ for_each_new_connector_in_state(state, connector, connector_state, i) {
if (connector_state->crtc != &crtc->base)
continue;
ret = skl_update_scaler_crtc(pipe_config);
if (!ret)
- ret = intel_atomic_setup_scalers(dev, intel_crtc,
+ ret = intel_atomic_setup_scalers(dev_priv, intel_crtc,
pipe_config);
}
static void intel_modeset_update_connector_atomic_state(struct drm_device *dev)
{
struct intel_connector *connector;
+ struct drm_connector_list_iter conn_iter;
- for_each_intel_connector(dev, connector) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ for_each_intel_connector_iter(connector, &conn_iter) {
if (connector->base.state->crtc)
drm_connector_unreference(&connector->base);
connector->base.state->crtc = NULL;
}
}
+ drm_connector_list_iter_end(&conn_iter);
}
static void
state = pipe_config->base.state;
/* Clamp display bpp to EDID value */
- for_each_connector_in_state(state, connector, connector_state, i) {
+ for_each_new_connector_in_state(state, connector, connector_state, i) {
if (connector_state->crtc != &crtc->base)
continue;
DRM_DEBUG_KMS("adjusted mode:\n");
drm_mode_debug_printmodeline(&pipe_config->base.adjusted_mode);
intel_dump_crtc_timings(&pipe_config->base.adjusted_mode);
- DRM_DEBUG_KMS("port clock: %d, pipe src size: %dx%d\n",
+ DRM_DEBUG_KMS("port clock: %d, pipe src size: %dx%d, pixel rate %d\n",
pipe_config->port_clock,
- pipe_config->pipe_src_w, pipe_config->pipe_src_h);
+ pipe_config->pipe_src_w, pipe_config->pipe_src_h,
+ pipe_config->pixel_rate);
if (INTEL_GEN(dev_priv) >= 9)
DRM_DEBUG_KMS("num_scalers: %d, scaler_users: 0x%x, scaler_id: %d\n",
static void
clear_intel_crtc_state(struct intel_crtc_state *crtc_state)
{
- struct drm_crtc_state tmp_state;
+ struct drm_i915_private *dev_priv =
+ to_i915(crtc_state->base.crtc->dev);
struct intel_crtc_scaler_state scaler_state;
struct intel_dpll_hw_state dpll_hw_state;
struct intel_shared_dpll *shared_dpll;
+ struct intel_crtc_wm_state wm_state;
bool force_thru;
/* FIXME: before the switch to atomic started, a new pipe_config was
* fixed, so that the crtc_state can be safely duplicated. For now,
* only fields that are know to not cause problems are preserved. */
- tmp_state = crtc_state->base;
scaler_state = crtc_state->scaler_state;
shared_dpll = crtc_state->shared_dpll;
dpll_hw_state = crtc_state->dpll_hw_state;
force_thru = crtc_state->pch_pfit.force_thru;
+ if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ wm_state = crtc_state->wm;
- memset(crtc_state, 0, sizeof *crtc_state);
+ /* Keep base drm_crtc_state intact, only clear our extended struct */
+ BUILD_BUG_ON(offsetof(struct intel_crtc_state, base));
+ memset(&crtc_state->base + 1, 0,
+ sizeof(*crtc_state) - sizeof(crtc_state->base));
- crtc_state->base = tmp_state;
crtc_state->scaler_state = scaler_state;
crtc_state->shared_dpll = shared_dpll;
crtc_state->dpll_hw_state = dpll_hw_state;
crtc_state->pch_pfit.force_thru = force_thru;
+ if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ crtc_state->wm = wm_state;
}
static int
&pipe_config->pipe_src_w,
&pipe_config->pipe_src_h);
- for_each_connector_in_state(state, connector, connector_state, i) {
+ for_each_new_connector_in_state(state, connector, connector_state, i) {
if (connector_state->crtc != crtc)
continue;
* adjust it according to limitations or connector properties, and also
* a chance to reject the mode entirely.
*/
- for_each_connector_in_state(state, connector, connector_state, i) {
+ for_each_new_connector_in_state(state, connector, connector_state, i) {
if (connector_state->crtc != crtc)
continue;
}
/* Dithering seems to not pass-through bits correctly when it should, so
- * only enable it on 6bpc panels. */
- pipe_config->dither = pipe_config->pipe_bpp == 6*3;
+ * only enable it on 6bpc panels and when its not a compliance
+ * test requesting 6bpc video pattern.
+ */
+ pipe_config->dither = (pipe_config->pipe_bpp == 6*3) &&
+ !pipe_config->dither_force_disable;
DRM_DEBUG_KMS("hw max bpp: %i, pipe bpp: %i, dithering: %i\n",
base_bpp, pipe_config->pipe_bpp, pipe_config->dither);
intel_modeset_update_crtc_state(struct drm_atomic_state *state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *new_crtc_state;
int i;
/* Double check state. */
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
- to_intel_crtc(crtc)->config = to_intel_crtc_state(crtc->state);
+ for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
+ to_intel_crtc(crtc)->config = to_intel_crtc_state(new_crtc_state);
/* Update hwmode for vblank functions */
- if (crtc->state->active)
- crtc->hwmode = crtc->state->adjusted_mode;
+ if (new_crtc_state->active)
+ crtc->hwmode = new_crtc_state->adjusted_mode;
else
crtc->hwmode.crtc_clock = 0;
}
PIPE_CONF_CHECK_I(scaler_state.scaler_id);
+ PIPE_CONF_CHECK_CLOCK_FUZZY(pixel_rate);
}
/* BDW+ don't expose a synchronous way to read the state */
struct drm_crtc *crtc)
{
struct drm_connector *connector;
- struct drm_connector_state *old_conn_state;
+ struct drm_connector_state *new_conn_state;
int i;
- for_each_connector_in_state(state, connector, old_conn_state, i) {
+ for_each_new_connector_in_state(state, connector, new_conn_state, i) {
struct drm_encoder *encoder = connector->encoder;
- struct drm_connector_state *state = connector->state;
- if (state->crtc != crtc)
+ if (new_conn_state->crtc != crtc)
continue;
intel_connector_verify_state(to_intel_connector(connector));
- I915_STATE_WARN(state->best_encoder != encoder,
+ I915_STATE_WARN(new_conn_state->best_encoder != encoder,
"connector's atomic encoder doesn't match legacy encoder\n");
}
}
static void
-verify_encoder_state(struct drm_device *dev)
+verify_encoder_state(struct drm_device *dev, struct drm_atomic_state *state)
{
struct intel_encoder *encoder;
- struct intel_connector *connector;
+ struct drm_connector *connector;
+ struct drm_connector_state *old_conn_state, *new_conn_state;
+ int i;
for_each_intel_encoder(dev, encoder) {
- bool enabled = false;
+ bool enabled = false, found = false;
enum pipe pipe;
DRM_DEBUG_KMS("[ENCODER:%d:%s]\n",
encoder->base.base.id,
encoder->base.name);
- for_each_intel_connector(dev, connector) {
- if (connector->base.state->best_encoder != &encoder->base)
+ for_each_oldnew_connector_in_state(state, connector, old_conn_state,
+ new_conn_state, i) {
+ if (old_conn_state->best_encoder == &encoder->base)
+ found = true;
+
+ if (new_conn_state->best_encoder != &encoder->base)
continue;
- enabled = true;
+ found = enabled = true;
- I915_STATE_WARN(connector->base.state->crtc !=
+ I915_STATE_WARN(new_conn_state->crtc !=
encoder->base.crtc,
"connector's crtc doesn't match encoder crtc\n");
}
+ if (!found)
+ continue;
+
I915_STATE_WARN(!!encoder->base.crtc != enabled,
"encoder's enabled state mismatch "
"(expected %i, found %i)\n",
}
}
+ intel_crtc_compute_pixel_rate(pipe_config);
+
if (!new_crtc_state->active)
return;
intel_modeset_verify_disabled(struct drm_device *dev,
struct drm_atomic_state *state)
{
- verify_encoder_state(dev);
+ verify_encoder_state(dev, state);
verify_connector_state(dev, state, NULL);
verify_disabled_dpll_state(dev);
}
struct drm_device *dev = state->dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
int i;
if (!dev_priv->display.crtc_compute_clock)
return;
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct intel_shared_dpll *old_dpll =
- to_intel_crtc_state(crtc->state)->shared_dpll;
+ to_intel_crtc_state(old_crtc_state)->shared_dpll;
- if (!needs_modeset(crtc_state))
+ if (!needs_modeset(new_crtc_state))
continue;
- to_intel_crtc_state(crtc_state)->shared_dpll = NULL;
+ to_intel_crtc_state(new_crtc_state)->shared_dpll = NULL;
if (!old_dpll)
continue;
int i;
/* look at all crtc's that are going to be enabled in during modeset */
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
intel_crtc = to_intel_crtc(crtc);
if (!crtc_state->active || !needs_modeset(crtc_state))
struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
struct drm_i915_private *dev_priv = to_i915(state->dev);
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
int ret = 0, i;
if (!check_digital_port_conflicts(state)) {
intel_state->modeset = true;
intel_state->active_crtcs = dev_priv->active_crtcs;
+ intel_state->cdclk.logical = dev_priv->cdclk.logical;
+ intel_state->cdclk.actual = dev_priv->cdclk.actual;
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
- if (crtc_state->active)
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
+ if (new_crtc_state->active)
intel_state->active_crtcs |= 1 << i;
else
intel_state->active_crtcs &= ~(1 << i);
- if (crtc_state->active != crtc->state->active)
+ if (old_crtc_state->active != new_crtc_state->active)
intel_state->active_pipe_changes |= drm_crtc_mask(crtc);
}
* adjusted_mode bits in the crtc directly.
*/
if (dev_priv->display.modeset_calc_cdclk) {
- if (!intel_state->cdclk_pll_vco)
- intel_state->cdclk_pll_vco = dev_priv->cdclk_pll.vco;
- if (!intel_state->cdclk_pll_vco)
- intel_state->cdclk_pll_vco = dev_priv->skl_preferred_vco_freq;
-
ret = dev_priv->display.modeset_calc_cdclk(state);
if (ret < 0)
return ret;
/*
- * Writes to dev_priv->atomic_cdclk_freq must protected by
+ * Writes to dev_priv->cdclk.logical must protected by
* holding all the crtc locks, even if we don't end up
* touching the hardware
*/
- if (intel_state->cdclk != dev_priv->atomic_cdclk_freq) {
+ if (!intel_cdclk_state_compare(&dev_priv->cdclk.logical,
+ &intel_state->cdclk.logical)) {
ret = intel_lock_all_pipes(state);
if (ret < 0)
return ret;
}
/* All pipes must be switched off while we change the cdclk. */
- if (intel_state->dev_cdclk != dev_priv->cdclk_freq ||
- intel_state->cdclk_pll_vco != dev_priv->cdclk_pll.vco) {
+ if (!intel_cdclk_state_compare(&dev_priv->cdclk.actual,
+ &intel_state->cdclk.actual)) {
ret = intel_modeset_all_pipes(state);
if (ret < 0)
return ret;
}
- DRM_DEBUG_KMS("New cdclk calculated to be atomic %u, actual %u\n",
- intel_state->cdclk, intel_state->dev_cdclk);
+ DRM_DEBUG_KMS("New cdclk calculated to be logical %u kHz, actual %u kHz\n",
+ intel_state->cdclk.logical.cdclk,
+ intel_state->cdclk.actual.cdclk);
} else {
- to_intel_atomic_state(state)->cdclk = dev_priv->atomic_cdclk_freq;
+ to_intel_atomic_state(state)->cdclk.logical = dev_priv->cdclk.logical;
}
intel_modeset_clear_plls(state);
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *old_crtc_state, *crtc_state;
int ret, i;
bool any_ms = false;
if (ret)
return ret;
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, crtc_state, i) {
struct intel_crtc_state *pipe_config =
to_intel_crtc_state(crtc_state);
/* Catch I915_MODE_FLAG_INHERITED */
- if (crtc_state->mode.private_flags != crtc->state->mode.private_flags)
+ if (crtc_state->mode.private_flags != old_crtc_state->mode.private_flags)
crtc_state->mode_changed = true;
if (!needs_modeset(crtc_state))
if (i915.fastboot &&
intel_pipe_config_compare(dev_priv,
- to_intel_crtc_state(crtc->state),
+ to_intel_crtc_state(old_crtc_state),
pipe_config, true)) {
crtc_state->mode_changed = false;
- to_intel_crtc_state(crtc_state)->update_pipe = true;
+ pipe_config->update_pipe = true;
}
if (needs_modeset(crtc_state))
if (ret)
return ret;
} else {
- intel_state->cdclk = dev_priv->atomic_cdclk_freq;
+ intel_state->cdclk.logical = dev_priv->cdclk.logical;
}
ret = drm_atomic_helper_check_planes(dev, state);
struct drm_crtc *crtc;
int i, ret;
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
if (state->legacy_cursor_update)
continue;
if (crtc_state->update_wm_post)
return true;
- /*
- * cxsr is re-enabled after vblank.
- * This is already handled by crtc_state->update_wm_post,
- * but added for clarity.
- */
- if (crtc_state->disable_cxsr)
+ if (crtc_state->wm.need_postvbl_update)
return true;
return false;
static void intel_update_crtc(struct drm_crtc *crtc,
struct drm_atomic_state *state,
struct drm_crtc_state *old_crtc_state,
+ struct drm_crtc_state *new_crtc_state,
unsigned int *crtc_vblank_mask)
{
struct drm_device *dev = crtc->dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- struct intel_crtc_state *pipe_config = to_intel_crtc_state(crtc->state);
- bool modeset = needs_modeset(crtc->state);
+ struct intel_crtc_state *pipe_config = to_intel_crtc_state(new_crtc_state);
+ bool modeset = needs_modeset(new_crtc_state);
if (modeset) {
update_scanline_offset(intel_crtc);
dev_priv->display.crtc_enable(pipe_config, state);
} else {
- intel_pre_plane_update(to_intel_crtc_state(old_crtc_state));
+ intel_pre_plane_update(to_intel_crtc_state(old_crtc_state),
+ pipe_config);
}
if (drm_atomic_get_existing_plane_state(state, crtc->primary)) {
unsigned int *crtc_vblank_mask)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *old_crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
int i;
- for_each_crtc_in_state(state, crtc, old_crtc_state, i) {
- if (!crtc->state->active)
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
+ if (!new_crtc_state->active)
continue;
intel_update_crtc(crtc, state, old_crtc_state,
- crtc_vblank_mask);
+ new_crtc_state, crtc_vblank_mask);
}
}
struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
struct drm_crtc *crtc;
struct intel_crtc *intel_crtc;
- struct drm_crtc_state *old_crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
struct intel_crtc_state *cstate;
unsigned int updated = 0;
bool progress;
const struct skl_ddb_entry *entries[I915_MAX_PIPES] = {};
- for_each_crtc_in_state(state, crtc, old_crtc_state, i)
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i)
/* ignore allocations for crtc's that have been turned off. */
- if (crtc->state->active)
+ if (new_crtc_state->active)
entries[i] = &to_intel_crtc_state(old_crtc_state)->wm.skl.ddb;
/*
do {
progress = false;
- for_each_crtc_in_state(state, crtc, old_crtc_state, i) {
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
bool vbl_wait = false;
unsigned int cmask = drm_crtc_mask(crtc);
*/
if (!skl_ddb_entry_equal(&cstate->wm.skl.ddb,
&to_intel_crtc_state(old_crtc_state)->wm.skl.ddb) &&
- !crtc->state->active_changed &&
+ !new_crtc_state->active_changed &&
intel_state->wm_results.dirty_pipes != updated)
vbl_wait = true;
intel_update_crtc(crtc, state, old_crtc_state,
- crtc_vblank_mask);
+ new_crtc_state, crtc_vblank_mask);
if (vbl_wait)
intel_wait_for_vblank(dev_priv, pipe);
struct drm_device *dev = state->dev;
struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
struct drm_i915_private *dev_priv = to_i915(dev);
- struct drm_crtc_state *old_crtc_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
struct drm_crtc *crtc;
struct intel_crtc_state *intel_cstate;
bool hw_check = intel_state->modeset;
- unsigned long put_domains[I915_MAX_PIPES] = {};
+ u64 put_domains[I915_MAX_PIPES] = {};
unsigned crtc_vblank_mask = 0;
int i;
if (intel_state->modeset)
intel_display_power_get(dev_priv, POWER_DOMAIN_MODESET);
- for_each_crtc_in_state(state, crtc, old_crtc_state, i) {
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- if (needs_modeset(crtc->state) ||
- to_intel_crtc_state(crtc->state)->update_pipe) {
+ if (needs_modeset(new_crtc_state) ||
+ to_intel_crtc_state(new_crtc_state)->update_pipe) {
hw_check = true;
put_domains[to_intel_crtc(crtc)->pipe] =
modeset_get_crtc_power_domains(crtc,
- to_intel_crtc_state(crtc->state));
+ to_intel_crtc_state(new_crtc_state));
}
- if (!needs_modeset(crtc->state))
+ if (!needs_modeset(new_crtc_state))
continue;
- intel_pre_plane_update(to_intel_crtc_state(old_crtc_state));
+ intel_pre_plane_update(to_intel_crtc_state(old_crtc_state),
+ to_intel_crtc_state(new_crtc_state));
if (old_crtc_state->active) {
intel_crtc_disable_planes(crtc, old_crtc_state->plane_mask);
/*
* Make sure we don't call initial_watermarks
* for ILK-style watermark updates.
+ *
+ * No clue what this is supposed to achieve.
*/
- if (dev_priv->display.atomic_update_watermarks)
+ if (INTEL_GEN(dev_priv) >= 9)
dev_priv->display.initial_watermarks(intel_state,
to_intel_crtc_state(crtc->state));
- else
- intel_update_watermarks(intel_crtc);
}
}
}
if (intel_state->modeset) {
drm_atomic_helper_update_legacy_modeset_state(state->dev, state);
- if (dev_priv->display.modeset_commit_cdclk &&
- (intel_state->dev_cdclk != dev_priv->cdclk_freq ||
- intel_state->cdclk_pll_vco != dev_priv->cdclk_pll.vco))
- dev_priv->display.modeset_commit_cdclk(state);
+ intel_set_cdclk(dev_priv, &dev_priv->cdclk.actual);
/*
* SKL workaround: bspec recommends we disable the SAGV when we
}
/* Complete the events for pipes that have now been disabled */
- for_each_crtc_in_state(state, crtc, old_crtc_state, i) {
- bool modeset = needs_modeset(crtc->state);
+ for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
+ bool modeset = needs_modeset(new_crtc_state);
/* Complete events for now disable pipes here. */
- if (modeset && !crtc->state->active && crtc->state->event) {
+ if (modeset && !new_crtc_state->active && new_crtc_state->event) {
spin_lock_irq(&dev->event_lock);
- drm_crtc_send_vblank_event(crtc, crtc->state->event);
+ drm_crtc_send_vblank_event(crtc, new_crtc_state->event);
spin_unlock_irq(&dev->event_lock);
- crtc->state->event = NULL;
+ new_crtc_state->event = NULL;
}
}
*
* TODO: Move this (and other cleanup) to an async worker eventually.
*/
- for_each_crtc_in_state(state, crtc, old_crtc_state, i) {
- intel_cstate = to_intel_crtc_state(crtc->state);
+ for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
+ intel_cstate = to_intel_crtc_state(new_crtc_state);
if (dev_priv->display.optimize_watermarks)
dev_priv->display.optimize_watermarks(intel_state,
intel_cstate);
}
- for_each_crtc_in_state(state, crtc, old_crtc_state, i) {
+ for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
intel_post_plane_update(to_intel_crtc_state(old_crtc_state));
if (put_domains[i])
modeset_put_power_domains(dev_priv, put_domains[i]);
- intel_modeset_verify_crtc(crtc, state, old_crtc_state, crtc->state);
+ intel_modeset_verify_crtc(crtc, state, old_crtc_state, new_crtc_state);
}
if (intel_state->modeset && intel_can_enable_sagv(state))
static void intel_atomic_track_fbs(struct drm_atomic_state *state)
{
- struct drm_plane_state *old_plane_state;
+ struct drm_plane_state *old_plane_state, *new_plane_state;
struct drm_plane *plane;
int i;
- for_each_plane_in_state(state, plane, old_plane_state, i)
+ for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i)
i915_gem_track_fb(intel_fb_obj(old_plane_state->fb),
- intel_fb_obj(plane->state->fb),
+ intel_fb_obj(new_plane_state->fb),
to_intel_plane(plane)->frontbuffer_bit);
}
struct drm_i915_private *dev_priv = to_i915(dev);
int ret = 0;
+ /*
+ * The intel_legacy_cursor_update() fast path takes care
+ * of avoiding the vblank waits for simple cursor
+ * movement and flips. For cursor on/off and size changes,
+ * we want to perform the vblank waits so that watermark
+ * updates happen during the correct frames. Gen9+ have
+ * double buffered watermarks and so shouldn't need this.
+ */
+ if (INTEL_GEN(dev_priv) < 9)
+ state->legacy_cursor_update = false;
+
ret = drm_atomic_helper_setup_commit(state, nonblock);
if (ret)
return ret;
memcpy(dev_priv->min_pixclk, intel_state->min_pixclk,
sizeof(intel_state->min_pixclk));
dev_priv->active_crtcs = intel_state->active_crtcs;
- dev_priv->atomic_cdclk_freq = intel_state->cdclk;
+ dev_priv->cdclk.logical = intel_state->cdclk.logical;
+ dev_priv->cdclk.actual = intel_state->cdclk.actual;
}
drm_atomic_state_get(state);
.set_config = drm_atomic_helper_set_config,
.set_property = drm_atomic_helper_crtc_set_property,
.destroy = intel_crtc_destroy,
- .page_flip = intel_crtc_page_flip,
+ .page_flip = drm_atomic_helper_page_flip,
.atomic_duplicate_state = intel_crtc_duplicate_state,
.atomic_destroy_state = intel_crtc_destroy_state,
.set_crc_source = intel_crtc_set_crc_source,
struct drm_i915_gem_object *old_obj = intel_fb_obj(plane->state->fb);
int ret;
+ if (obj) {
+ if (plane->type == DRM_PLANE_TYPE_CURSOR &&
+ INTEL_INFO(dev_priv)->cursor_needs_physical) {
+ const int align = IS_I830(dev_priv) ? 16 * 1024 : 256;
+
+ ret = i915_gem_object_attach_phys(obj, align);
+ if (ret) {
+ DRM_DEBUG_KMS("failed to attach phys object\n");
+ return ret;
+ }
+ } else {
+ struct i915_vma *vma;
+
+ vma = intel_pin_and_fence_fb_obj(fb, new_state->rotation);
+ if (IS_ERR(vma)) {
+ DRM_DEBUG_KMS("failed to pin object\n");
+ return PTR_ERR(vma);
+ }
+
+ to_intel_plane_state(new_state)->vma = vma;
+ }
+ }
+
if (!obj && !old_obj)
return 0;
i915_gem_object_wait_priority(obj, 0, I915_PRIORITY_DISPLAY);
}
- if (plane->type == DRM_PLANE_TYPE_CURSOR &&
- INTEL_INFO(dev_priv)->cursor_needs_physical) {
- int align = IS_I830(dev_priv) ? 16 * 1024 : 256;
- ret = i915_gem_object_attach_phys(obj, align);
- if (ret) {
- DRM_DEBUG_KMS("failed to attach phys object\n");
- return ret;
- }
- } else {
- struct i915_vma *vma;
-
- vma = intel_pin_and_fence_fb_obj(fb, new_state->rotation);
- if (IS_ERR(vma)) {
- DRM_DEBUG_KMS("failed to pin object\n");
- return PTR_ERR(vma);
- }
-
- to_intel_plane_state(new_state)->vma = vma;
- }
-
return 0;
}
int
skl_max_scale(struct intel_crtc *intel_crtc, struct intel_crtc_state *crtc_state)
{
+ struct drm_i915_private *dev_priv;
int max_scale;
- int crtc_clock, cdclk;
+ int crtc_clock, max_dotclk;
if (!intel_crtc || !crtc_state->base.enable)
return DRM_PLANE_HELPER_NO_SCALING;
+ dev_priv = to_i915(intel_crtc->base.dev);
+
crtc_clock = crtc_state->base.adjusted_mode.crtc_clock;
- cdclk = to_intel_atomic_state(crtc_state->base.state)->cdclk;
+ max_dotclk = to_intel_atomic_state(crtc_state->base.state)->cdclk.logical.cdclk;
- if (WARN_ON_ONCE(!crtc_clock || cdclk < crtc_clock))
+ if (IS_GEMINILAKE(dev_priv))
+ max_dotclk *= 2;
+
+ if (WARN_ON_ONCE(!crtc_clock || max_dotclk < crtc_clock))
return DRM_PLANE_HELPER_NO_SCALING;
/*
* or
* cdclk/crtc_clock
*/
- max_scale = min((1 << 16) * 3 - 1, (1 << 8) * ((cdclk << 8) / crtc_clock));
+ max_scale = min((1 << 16) * 3 - 1,
+ (1 << 8) * ((max_dotclk << 8) / crtc_clock));
return max_scale;
}
old_plane_state->src_h != src_h ||
old_plane_state->crtc_w != crtc_w ||
old_plane_state->crtc_h != crtc_h ||
- !old_plane_state->visible ||
- old_plane_state->fb->modifier != fb->modifier)
+ !old_plane_state->fb != !fb)
goto slow;
new_plane_state = intel_plane_duplicate_state(plane);
if (ret)
goto out_free;
- /* Visibility changed, must take slowpath. */
- if (!new_plane_state->visible)
- goto slow_free;
-
ret = mutex_lock_interruptible(&dev_priv->drm.struct_mutex);
if (ret)
goto out_free;
new_plane_state->fb = old_fb;
to_intel_plane_state(new_plane_state)->vma = old_vma;
- intel_plane->update_plane(plane,
- to_intel_crtc_state(crtc->state),
- to_intel_plane_state(plane->state));
+ if (plane->state->visible) {
+ trace_intel_update_plane(plane, to_intel_crtc(crtc));
+ intel_plane->update_plane(plane,
+ to_intel_crtc_state(crtc->state),
+ to_intel_plane_state(plane->state));
+ } else {
+ trace_intel_disable_plane(plane, to_intel_crtc(crtc));
+ intel_plane->disable_plane(plane, crtc);
+ }
intel_cleanup_plane_fb(plane, new_plane_state);
intel_plane_destroy_state(plane, new_plane_state);
return ret;
-slow_free:
- intel_plane_destroy_state(plane, new_plane_state);
slow:
return drm_atomic_helper_update_plane(plane, crtc, fb,
crtc_x, crtc_y, crtc_w, crtc_h,
intel_crtc->cursor_cntl = ~0;
intel_crtc->cursor_size = ~0;
- intel_crtc->wm.cxsr_allowed = true;
-
/* initialize shared scalers */
intel_crtc_init_scalers(intel_crtc, crtc_state);
enum pipe intel_get_pipe_from_connector(struct intel_connector *connector)
{
- struct drm_encoder *encoder = connector->base.encoder;
struct drm_device *dev = connector->base.dev;
WARN_ON(!drm_modeset_is_locked(&dev->mode_config.connection_mutex));
- if (!encoder || WARN_ON(!encoder->crtc))
+ if (!connector->base.state->crtc)
return INVALID_PIPE;
- return to_intel_crtc(encoder->crtc)->pipe;
+ return to_intel_crtc(connector->base.state->crtc)->pipe;
}
int intel_get_pipe_from_crtc_id(struct drm_device *dev, void *data,
*/
found = I915_READ(DDI_BUF_CTL(PORT_A)) & DDI_INIT_DISPLAY_DETECTED;
/* WaIgnoreDDIAStrap: skl */
- if (found || IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ if (found || IS_GEN9_BC(dev_priv))
intel_ddi_init(dev_priv, PORT_A);
/* DDI B, C and D detection is indicated by the SFUSE_STRAP
/*
* On SKL we don't have a way to detect DDI-E so we rely on VBT.
*/
- if ((IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) &&
+ if (IS_GEN9_BC(dev_priv) &&
(dev_priv->vbt.ddi_port_info[PORT_E].supports_dp ||
dev_priv->vbt.ddi_port_info[PORT_E].supports_dvi ||
dev_priv->vbt.ddi_port_info[PORT_E].supports_hdmi))
static void intel_user_framebuffer_destroy(struct drm_framebuffer *fb)
{
- struct drm_device *dev = fb->dev;
struct intel_framebuffer *intel_fb = to_intel_framebuffer(fb);
drm_framebuffer_cleanup(fb);
- mutex_lock(&dev->struct_mutex);
+
+ i915_gem_object_lock(intel_fb->obj);
WARN_ON(!intel_fb->obj->framebuffer_references--);
+ i915_gem_object_unlock(intel_fb->obj);
+
i915_gem_object_put(intel_fb->obj);
- mutex_unlock(&dev->struct_mutex);
+
kfree(intel_fb);
}
struct drm_clip_rect *clips,
unsigned num_clips)
{
- struct drm_device *dev = fb->dev;
- struct intel_framebuffer *intel_fb = to_intel_framebuffer(fb);
- struct drm_i915_gem_object *obj = intel_fb->obj;
+ struct drm_i915_gem_object *obj = intel_fb_obj(fb);
- mutex_lock(&dev->struct_mutex);
- if (obj->pin_display && obj->cache_dirty)
- i915_gem_clflush_object(obj, true);
- intel_fb_obj_flush(obj, false, ORIGIN_DIRTYFB);
- mutex_unlock(&dev->struct_mutex);
+ i915_gem_object_flush_if_display(obj);
+ intel_fb_obj_flush(obj, ORIGIN_DIRTYFB);
return 0;
}
u32 intel_fb_pitch_limit(struct drm_i915_private *dev_priv,
uint64_t fb_modifier, uint32_t pixel_format)
{
- u32 gen = INTEL_INFO(dev_priv)->gen;
+ u32 gen = INTEL_GEN(dev_priv);
if (gen >= 9) {
int cpp = drm_format_plane_cpp(pixel_format, 0);
* pixels and 32K bytes."
*/
return min(8192 * cpp, 32768);
- } else if (gen >= 5 && !IS_VALLEYVIEW(dev_priv) &&
- !IS_CHERRYVIEW(dev_priv)) {
+ } else if (gen >= 5 && !HAS_GMCH_DISPLAY(dev_priv)) {
return 32*1024;
} else if (gen >= 4) {
if (fb_modifier == I915_FORMAT_MOD_X_TILED)
}
}
-static int intel_framebuffer_init(struct drm_device *dev,
- struct intel_framebuffer *intel_fb,
- struct drm_mode_fb_cmd2 *mode_cmd,
- struct drm_i915_gem_object *obj)
+static int intel_framebuffer_init(struct intel_framebuffer *intel_fb,
+ struct drm_i915_gem_object *obj,
+ struct drm_mode_fb_cmd2 *mode_cmd)
{
- struct drm_i915_private *dev_priv = to_i915(dev);
- unsigned int tiling = i915_gem_object_get_tiling(obj);
- int ret;
- u32 pitch_limit, stride_alignment;
+ struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
struct drm_format_name_buf format_name;
+ u32 pitch_limit, stride_alignment;
+ unsigned int tiling, stride;
+ int ret = -EINVAL;
- WARN_ON(!mutex_is_locked(&dev->struct_mutex));
+ i915_gem_object_lock(obj);
+ obj->framebuffer_references++;
+ tiling = i915_gem_object_get_tiling(obj);
+ stride = i915_gem_object_get_stride(obj);
+ i915_gem_object_unlock(obj);
if (mode_cmd->flags & DRM_MODE_FB_MODIFIERS) {
/*
*/
if (tiling != I915_TILING_NONE &&
tiling != intel_fb_modifier_to_tiling(mode_cmd->modifier[0])) {
- DRM_DEBUG("tiling_mode doesn't match fb modifier\n");
- return -EINVAL;
+ DRM_DEBUG_KMS("tiling_mode doesn't match fb modifier\n");
+ goto err;
}
} else {
if (tiling == I915_TILING_X) {
mode_cmd->modifier[0] = I915_FORMAT_MOD_X_TILED;
} else if (tiling == I915_TILING_Y) {
- DRM_DEBUG("No Y tiling for legacy addfb\n");
- return -EINVAL;
+ DRM_DEBUG_KMS("No Y tiling for legacy addfb\n");
+ goto err;
}
}
case I915_FORMAT_MOD_Y_TILED:
case I915_FORMAT_MOD_Yf_TILED:
if (INTEL_GEN(dev_priv) < 9) {
- DRM_DEBUG("Unsupported tiling 0x%llx!\n",
- mode_cmd->modifier[0]);
- return -EINVAL;
+ DRM_DEBUG_KMS("Unsupported tiling 0x%llx!\n",
+ mode_cmd->modifier[0]);
+ goto err;
}
case DRM_FORMAT_MOD_NONE:
case I915_FORMAT_MOD_X_TILED:
break;
default:
- DRM_DEBUG("Unsupported fb modifier 0x%llx!\n",
- mode_cmd->modifier[0]);
- return -EINVAL;
+ DRM_DEBUG_KMS("Unsupported fb modifier 0x%llx!\n",
+ mode_cmd->modifier[0]);
+ goto err;
}
/*
*/
if (INTEL_INFO(dev_priv)->gen < 4 &&
tiling != intel_fb_modifier_to_tiling(mode_cmd->modifier[0])) {
- DRM_DEBUG("tiling_mode must match fb modifier exactly on gen2/3\n");
- return -EINVAL;
- }
-
- stride_alignment = intel_fb_stride_alignment(dev_priv,
- mode_cmd->modifier[0],
- mode_cmd->pixel_format);
- if (mode_cmd->pitches[0] & (stride_alignment - 1)) {
- DRM_DEBUG("pitch (%d) must be at least %u byte aligned\n",
- mode_cmd->pitches[0], stride_alignment);
- return -EINVAL;
+ DRM_DEBUG_KMS("tiling_mode must match fb modifier exactly on gen2/3\n");
+ goto err;
}
pitch_limit = intel_fb_pitch_limit(dev_priv, mode_cmd->modifier[0],
mode_cmd->pixel_format);
if (mode_cmd->pitches[0] > pitch_limit) {
- DRM_DEBUG("%s pitch (%u) must be at less than %d\n",
- mode_cmd->modifier[0] != DRM_FORMAT_MOD_NONE ?
- "tiled" : "linear",
- mode_cmd->pitches[0], pitch_limit);
- return -EINVAL;
+ DRM_DEBUG_KMS("%s pitch (%u) must be at most %d\n",
+ mode_cmd->modifier[0] != DRM_FORMAT_MOD_NONE ?
+ "tiled" : "linear",
+ mode_cmd->pitches[0], pitch_limit);
+ goto err;
}
/*
* If there's a fence, enforce that
* the fb pitch and fence stride match.
*/
- if (tiling != I915_TILING_NONE &&
- mode_cmd->pitches[0] != i915_gem_object_get_stride(obj)) {
- DRM_DEBUG("pitch (%d) must match tiling stride (%d)\n",
- mode_cmd->pitches[0],
- i915_gem_object_get_stride(obj));
- return -EINVAL;
+ if (tiling != I915_TILING_NONE && mode_cmd->pitches[0] != stride) {
+ DRM_DEBUG_KMS("pitch (%d) must match tiling stride (%d)\n",
+ mode_cmd->pitches[0], stride);
+ goto err;
}
/* Reject formats not supported by any plane early. */
break;
case DRM_FORMAT_XRGB1555:
if (INTEL_GEN(dev_priv) > 3) {
- DRM_DEBUG("unsupported pixel format: %s\n",
- drm_get_format_name(mode_cmd->pixel_format, &format_name));
- return -EINVAL;
+ DRM_DEBUG_KMS("unsupported pixel format: %s\n",
+ drm_get_format_name(mode_cmd->pixel_format, &format_name));
+ goto err;
}
break;
case DRM_FORMAT_ABGR8888:
if (!IS_VALLEYVIEW(dev_priv) && !IS_CHERRYVIEW(dev_priv) &&
INTEL_GEN(dev_priv) < 9) {
- DRM_DEBUG("unsupported pixel format: %s\n",
- drm_get_format_name(mode_cmd->pixel_format, &format_name));
- return -EINVAL;
+ DRM_DEBUG_KMS("unsupported pixel format: %s\n",
+ drm_get_format_name(mode_cmd->pixel_format, &format_name));
+ goto err;
}
break;
case DRM_FORMAT_XBGR8888:
case DRM_FORMAT_XRGB2101010:
case DRM_FORMAT_XBGR2101010:
if (INTEL_GEN(dev_priv) < 4) {
- DRM_DEBUG("unsupported pixel format: %s\n",
- drm_get_format_name(mode_cmd->pixel_format, &format_name));
- return -EINVAL;
+ DRM_DEBUG_KMS("unsupported pixel format: %s\n",
+ drm_get_format_name(mode_cmd->pixel_format, &format_name));
+ goto err;
}
break;
case DRM_FORMAT_ABGR2101010:
if (!IS_VALLEYVIEW(dev_priv) && !IS_CHERRYVIEW(dev_priv)) {
- DRM_DEBUG("unsupported pixel format: %s\n",
- drm_get_format_name(mode_cmd->pixel_format, &format_name));
- return -EINVAL;
+ DRM_DEBUG_KMS("unsupported pixel format: %s\n",
+ drm_get_format_name(mode_cmd->pixel_format, &format_name));
+ goto err;
}
break;
case DRM_FORMAT_YUYV:
case DRM_FORMAT_YVYU:
case DRM_FORMAT_VYUY:
if (INTEL_GEN(dev_priv) < 5) {
- DRM_DEBUG("unsupported pixel format: %s\n",
- drm_get_format_name(mode_cmd->pixel_format, &format_name));
- return -EINVAL;
+ DRM_DEBUG_KMS("unsupported pixel format: %s\n",
+ drm_get_format_name(mode_cmd->pixel_format, &format_name));
+ goto err;
}
break;
default:
- DRM_DEBUG("unsupported pixel format: %s\n",
- drm_get_format_name(mode_cmd->pixel_format, &format_name));
- return -EINVAL;
+ DRM_DEBUG_KMS("unsupported pixel format: %s\n",
+ drm_get_format_name(mode_cmd->pixel_format, &format_name));
+ goto err;
}
/* FIXME need to adjust LINOFF/TILEOFF accordingly. */
if (mode_cmd->offsets[0] != 0)
- return -EINVAL;
+ goto err;
+
+ drm_helper_mode_fill_fb_struct(&dev_priv->drm,
+ &intel_fb->base, mode_cmd);
+
+ stride_alignment = intel_fb_stride_alignment(&intel_fb->base, 0);
+ if (mode_cmd->pitches[0] & (stride_alignment - 1)) {
+ DRM_DEBUG_KMS("pitch (%d) must be at least %u byte aligned\n",
+ mode_cmd->pitches[0], stride_alignment);
+ goto err;
+ }
- drm_helper_mode_fill_fb_struct(dev, &intel_fb->base, mode_cmd);
intel_fb->obj = obj;
ret = intel_fill_fb_info(dev_priv, &intel_fb->base);
if (ret)
- return ret;
+ goto err;
- ret = drm_framebuffer_init(dev, &intel_fb->base, &intel_fb_funcs);
+ ret = drm_framebuffer_init(obj->base.dev,
+ &intel_fb->base,
+ &intel_fb_funcs);
if (ret) {
DRM_ERROR("framebuffer init failed %d\n", ret);
- return ret;
+ goto err;
}
- intel_fb->obj->framebuffer_references++;
-
return 0;
+
+err:
+ i915_gem_object_lock(obj);
+ obj->framebuffer_references--;
+ i915_gem_object_unlock(obj);
+ return ret;
}
static struct drm_framebuffer *
if (!obj)
return ERR_PTR(-ENOENT);
- fb = intel_framebuffer_create(dev, &mode_cmd, obj);
+ fb = intel_framebuffer_create(obj, &mode_cmd);
if (IS_ERR(fb))
i915_gem_object_put(obj);
*/
void intel_init_display_hooks(struct drm_i915_private *dev_priv)
{
+ intel_init_cdclk_hooks(dev_priv);
+
if (INTEL_INFO(dev_priv)->gen >= 9) {
dev_priv->display.get_pipe_config = haswell_get_pipe_config;
dev_priv->display.get_initial_plane_config =
dev_priv->display.crtc_disable = i9xx_crtc_disable;
}
- /* Returns the core display clock speed */
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
- dev_priv->display.get_display_clock_speed =
- skylake_get_display_clock_speed;
- else if (IS_GEN9_LP(dev_priv))
- dev_priv->display.get_display_clock_speed =
- broxton_get_display_clock_speed;
- else if (IS_BROADWELL(dev_priv))
- dev_priv->display.get_display_clock_speed =
- broadwell_get_display_clock_speed;
- else if (IS_HASWELL(dev_priv))
- dev_priv->display.get_display_clock_speed =
- haswell_get_display_clock_speed;
- else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
- dev_priv->display.get_display_clock_speed =
- valleyview_get_display_clock_speed;
- else if (IS_GEN5(dev_priv))
- dev_priv->display.get_display_clock_speed =
- ilk_get_display_clock_speed;
- else if (IS_I945G(dev_priv) || IS_I965G(dev_priv) ||
- IS_GEN6(dev_priv) || IS_IVYBRIDGE(dev_priv))
- dev_priv->display.get_display_clock_speed =
- i945_get_display_clock_speed;
- else if (IS_GM45(dev_priv))
- dev_priv->display.get_display_clock_speed =
- gm45_get_display_clock_speed;
- else if (IS_I965GM(dev_priv))
- dev_priv->display.get_display_clock_speed =
- i965gm_get_display_clock_speed;
- else if (IS_PINEVIEW(dev_priv))
- dev_priv->display.get_display_clock_speed =
- pnv_get_display_clock_speed;
- else if (IS_G33(dev_priv) || IS_G4X(dev_priv))
- dev_priv->display.get_display_clock_speed =
- g33_get_display_clock_speed;
- else if (IS_I915G(dev_priv))
- dev_priv->display.get_display_clock_speed =
- i915_get_display_clock_speed;
- else if (IS_I945GM(dev_priv) || IS_I845G(dev_priv))
- dev_priv->display.get_display_clock_speed =
- i9xx_misc_get_display_clock_speed;
- else if (IS_I915GM(dev_priv))
- dev_priv->display.get_display_clock_speed =
- i915gm_get_display_clock_speed;
- else if (IS_I865G(dev_priv))
- dev_priv->display.get_display_clock_speed =
- i865_get_display_clock_speed;
- else if (IS_I85X(dev_priv))
- dev_priv->display.get_display_clock_speed =
- i85x_get_display_clock_speed;
- else { /* 830 */
- WARN(!IS_I830(dev_priv), "Unknown platform. Assuming 133 MHz CDCLK\n");
- dev_priv->display.get_display_clock_speed =
- i830_get_display_clock_speed;
- }
-
if (IS_GEN5(dev_priv)) {
dev_priv->display.fdi_link_train = ironlake_fdi_link_train;
} else if (IS_GEN6(dev_priv)) {
dev_priv->display.fdi_link_train = hsw_fdi_link_train;
}
- if (IS_BROADWELL(dev_priv)) {
- dev_priv->display.modeset_commit_cdclk =
- broadwell_modeset_commit_cdclk;
- dev_priv->display.modeset_calc_cdclk =
- broadwell_modeset_calc_cdclk;
- } else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
- dev_priv->display.modeset_commit_cdclk =
- valleyview_modeset_commit_cdclk;
- dev_priv->display.modeset_calc_cdclk =
- valleyview_modeset_calc_cdclk;
- } else if (IS_GEN9_LP(dev_priv)) {
- dev_priv->display.modeset_commit_cdclk =
- bxt_modeset_commit_cdclk;
- dev_priv->display.modeset_calc_cdclk =
- bxt_modeset_calc_cdclk;
- } else if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
- dev_priv->display.modeset_commit_cdclk =
- skl_modeset_commit_cdclk;
- dev_priv->display.modeset_calc_cdclk =
- skl_modeset_calc_cdclk;
- }
-
if (dev_priv->info.gen >= 9)
dev_priv->display.update_crtcs = skl_update_crtcs;
else
struct drm_i915_private *dev_priv = to_i915(dev);
intel_update_cdclk(dev_priv);
-
- dev_priv->atomic_cdclk_freq = dev_priv->cdclk_freq;
+ dev_priv->cdclk.logical = dev_priv->cdclk.actual = dev_priv->cdclk.hw;
intel_init_clock_gating(dev_priv);
}
* intermediate watermarks (since we don't trust the current
* watermarks).
*/
- intel_state->skip_intermediate_wm = true;
+ if (!HAS_GMCH_DISPLAY(dev_priv))
+ intel_state->skip_intermediate_wm = true;
ret = intel_atomic_check(dev, state);
if (ret) {
}
/* Write calculated watermark values back */
- for_each_crtc_in_state(state, crtc, cstate, i) {
+ for_each_new_crtc_in_state(state, crtc, cstate, i) {
struct intel_crtc_state *cs = to_intel_crtc_state(cstate);
cs->wm.need_postvbl_update = true;
* Note that we need to do this after reconstructing the BIOS fb's
* since the watermark calculation done here will use pstate->fb.
*/
- sanitize_watermarks(dev);
+ if (!HAS_GMCH_DISPLAY(dev_priv))
+ sanitize_watermarks(dev);
return 0;
}
static void intel_enable_pipe_a(struct drm_device *dev)
{
struct intel_connector *connector;
+ struct drm_connector_list_iter conn_iter;
struct drm_connector *crt = NULL;
struct intel_load_detect_pipe load_detect_temp;
struct drm_modeset_acquire_ctx *ctx = dev->mode_config.acquire_ctx;
/* We can't just switch on the pipe A, we need to set things up with a
* proper mode and output configuration. As a gross hack, enable pipe A
* by enabling the load detect pipe once. */
- for_each_intel_connector(dev, connector) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ for_each_intel_connector_iter(connector, &conn_iter) {
if (connector->encoder->type == INTEL_OUTPUT_ANALOG) {
crt = &connector->base;
break;
}
}
+ drm_connector_list_iter_end(&conn_iter);
if (!crt)
return;
if (plane->base.type == DRM_PLANE_TYPE_PRIMARY)
continue;
+ trace_intel_disable_plane(&plane->base, crtc);
plane->disable_plane(&plane->base, &crtc->base);
}
}
/* FIXME read out full plane state for all planes */
static void readout_plane_state(struct intel_crtc *crtc)
{
- struct drm_plane *primary = crtc->base.primary;
- struct intel_plane_state *plane_state =
- to_intel_plane_state(primary->state);
+ struct intel_plane *primary = to_intel_plane(crtc->base.primary);
+ bool visible;
- plane_state->base.visible = crtc->active &&
- primary_get_hw_state(to_intel_plane(primary));
+ visible = crtc->active && primary_get_hw_state(primary);
- if (plane_state->base.visible)
- crtc->base.state->plane_mask |= 1 << drm_plane_index(primary);
+ intel_set_plane_visible(to_intel_crtc_state(crtc->base.state),
+ to_intel_plane_state(primary->base.state),
+ visible);
}
static void intel_modeset_readout_hw_state(struct drm_device *dev)
struct intel_crtc *crtc;
struct intel_encoder *encoder;
struct intel_connector *connector;
+ struct drm_connector_list_iter conn_iter;
int i;
dev_priv->active_crtcs = 0;
pipe_name(pipe));
}
- for_each_intel_connector(dev, connector) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ for_each_intel_connector_iter(connector, &conn_iter) {
if (connector->get_hw_state(connector)) {
connector->base.dpms = DRM_MODE_DPMS_ON;
connector->base.base.id, connector->base.name,
enableddisabled(connector->base.encoder));
}
+ drm_connector_list_iter_end(&conn_iter);
for_each_intel_crtc(dev, crtc) {
struct intel_crtc_state *crtc_state =
*/
crtc_state->base.mode.private_flags = I915_MODE_FLAG_INHERITED;
- if (INTEL_GEN(dev_priv) >= 9 || IS_BROADWELL(dev_priv))
- pixclk = ilk_pipe_pixel_rate(crtc_state);
- else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
- pixclk = crtc_state->base.adjusted_mode.crtc_clock;
+ intel_crtc_compute_pixel_rate(crtc_state);
+
+ if (INTEL_GEN(dev_priv) >= 9 || IS_BROADWELL(dev_priv) ||
+ IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ pixclk = crtc_state->pixel_rate;
else
WARN_ON(dev_priv->display.modeset_calc_cdclk);
}
}
+static void
+get_encoder_power_domains(struct drm_i915_private *dev_priv)
+{
+ struct intel_encoder *encoder;
+
+ for_each_intel_encoder(&dev_priv->drm, encoder) {
+ u64 get_domains;
+ enum intel_display_power_domain domain;
+
+ if (!encoder->get_power_domains)
+ continue;
+
+ get_domains = encoder->get_power_domains(encoder);
+ for_each_power_domain(domain, get_domains)
+ intel_display_power_get(dev_priv, domain);
+ }
+}
+
/* Scan out the current hw modeset state,
* and sanitizes it to the current state
*/
intel_modeset_readout_hw_state(dev);
/* HW state is read out, now we need to sanitize this mess. */
+ get_encoder_power_domains(dev_priv);
+
for_each_intel_encoder(dev, encoder) {
intel_sanitize_encoder(encoder);
}
pll->on = false;
}
- if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
vlv_wm_get_hw_state(dev);
- else if (IS_GEN9(dev_priv))
+ vlv_wm_sanitize(dev_priv);
+ } else if (IS_GEN9(dev_priv)) {
skl_wm_get_hw_state(dev);
- else if (HAS_PCH_SPLIT(dev_priv))
+ } else if (HAS_PCH_SPLIT(dev_priv)) {
ilk_wm_get_hw_state(dev);
+ }
for_each_intel_crtc(dev, crtc) {
- unsigned long put_domains;
+ u64 put_domains;
put_domains = modeset_get_crtc_power_domains(&crtc->base, crtc->config);
if (WARN_ON(put_domains))
}
intel_display_set_init_power(dev_priv, false);
+ intel_power_domains_verify_state(dev_priv);
+
intel_fbc_init_pipe_state(dev_priv);
}
}
if (!ret)
- ret = __intel_display_resume(dev, state);
+ ret = __intel_display_resume(dev, state, &ctx);
drm_modeset_drop_locks(&ctx);
drm_modeset_acquire_fini(&ctx);
void
intel_display_print_error_state(struct drm_i915_error_state_buf *m,
- struct drm_i915_private *dev_priv,
struct intel_display_error_state *error)
{
+ struct drm_i915_private *dev_priv = m->i915;
int i;
if (!error)
#include <linux/i2c.h>
#include <linux/slab.h>
#include <linux/export.h>
+#include <linux/types.h>
#include <linux/notifier.h>
#include <linux/reboot.h>
+#include <asm/byteorder.h>
#include <drm/drmP.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_crtc.h>
if (IS_GEN9_LP(dev_priv)) {
*source_rates = bxt_rates;
size = ARRAY_SIZE(bxt_rates);
- } else if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ } else if (IS_GEN9_BC(dev_priv)) {
*source_rates = skl_rates;
size = ARRAY_SIZE(skl_rates);
} else {
struct intel_encoder *encoder = &intel_dig_port->base;
struct drm_device *dev = encoder->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- enum intel_display_power_domain power_domain;
/*
* See vlv_power_sequencer_reset() why we need
* a power domain reference here.
*/
- power_domain = intel_display_port_aux_power_domain(encoder);
- intel_display_power_get(dev_priv, power_domain);
+ intel_display_power_get(dev_priv, intel_dp->aux_power_domain);
mutex_lock(&dev_priv->pps_mutex);
}
struct intel_encoder *encoder = &intel_dig_port->base;
struct drm_device *dev = encoder->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- enum intel_display_power_domain power_domain;
mutex_unlock(&dev_priv->pps_mutex);
- power_domain = intel_display_port_aux_power_domain(encoder);
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, intel_dp->aux_power_domain);
}
static void
* divide by 2000 and use that
*/
if (intel_dig_port->port == PORT_A)
- return DIV_ROUND_CLOSEST(dev_priv->cdclk_freq, 2000);
+ return DIV_ROUND_CLOSEST(dev_priv->cdclk.hw.cdclk, 2000);
else
return DIV_ROUND_CLOSEST(dev_priv->rawclk_freq, 2000);
}
if (bpc > 0)
bpp = min(bpp, 3*bpc);
+ /* For DP Compliance we override the computed bpp for the pipe */
+ if (intel_dp->compliance.test_data.bpc != 0) {
+ pipe_config->pipe_bpp = 3*intel_dp->compliance.test_data.bpc;
+ pipe_config->dither_force_disable = pipe_config->pipe_bpp == 6*3;
+ DRM_DEBUG_KMS("Setting pipe_bpp to %d\n",
+ pipe_config->pipe_bpp);
+ }
return bpp;
}
/* Conveniently, the link BW constants become indices with a shift...*/
int min_clock = 0;
int max_clock;
+ int link_rate_index;
int bpp, mode_rate;
int link_avail, link_clock;
int common_rates[DP_MAX_SUPPORTED_RATES] = {};
if (adjusted_mode->flags & DRM_MODE_FLAG_DBLCLK)
return false;
+ /* Use values requested by Compliance Test Request */
+ if (intel_dp->compliance.test_type == DP_TEST_LINK_TRAINING) {
+ link_rate_index = intel_dp_link_rate_index(intel_dp,
+ common_rates,
+ intel_dp->compliance.test_link_rate);
+ if (link_rate_index >= 0)
+ min_clock = max_clock = link_rate_index;
+ min_lane_count = max_lane_count = intel_dp->compliance.test_lane_count;
+ }
DRM_DEBUG_KMS("DP link computation with max lane count %i "
"max bw %d pixel clock %iKHz\n",
max_lane_count, common_rates[max_clock],
* DPLL0 VCO may need to be adjusted to get the correct
* clock for eDP. This will affect cdclk as well.
*/
- if (is_edp(intel_dp) &&
- (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))) {
+ if (is_edp(intel_dp) && IS_GEN9_BC(dev_priv)) {
int vco;
switch (pipe_config->port_clock / 2) {
break;
}
- to_intel_atomic_state(pipe_config->base.state)->cdclk_pll_vco = vco;
+ to_intel_atomic_state(pipe_config->base.state)->cdclk.logical.vco = vco;
}
if (!HAS_DDI(dev_priv))
{
struct drm_device *dev = intel_dp_to_dev(intel_dp);
struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
- struct intel_encoder *intel_encoder = &intel_dig_port->base;
struct drm_i915_private *dev_priv = to_i915(dev);
- enum intel_display_power_domain power_domain;
u32 pp;
i915_reg_t pp_stat_reg, pp_ctrl_reg;
bool need_to_disable = !intel_dp->want_panel_vdd;
if (edp_have_panel_vdd(intel_dp))
return need_to_disable;
- power_domain = intel_display_port_aux_power_domain(intel_encoder);
- intel_display_power_get(dev_priv, power_domain);
+ intel_display_power_get(dev_priv, intel_dp->aux_power_domain);
DRM_DEBUG_KMS("Turning eDP port %c VDD on\n",
port_name(intel_dig_port->port));
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_digital_port *intel_dig_port =
dp_to_dig_port(intel_dp);
- struct intel_encoder *intel_encoder = &intel_dig_port->base;
- enum intel_display_power_domain power_domain;
u32 pp;
i915_reg_t pp_stat_reg, pp_ctrl_reg;
if ((pp & PANEL_POWER_ON) == 0)
intel_dp->panel_power_off_time = ktime_get_boottime();
- power_domain = intel_display_port_aux_power_domain(intel_encoder);
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, intel_dp->aux_power_domain);
}
static void edp_panel_vdd_work(struct work_struct *__work)
static void edp_panel_off(struct intel_dp *intel_dp)
{
- struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
- struct intel_encoder *intel_encoder = &intel_dig_port->base;
struct drm_device *dev = intel_dp_to_dev(intel_dp);
struct drm_i915_private *dev_priv = to_i915(dev);
- enum intel_display_power_domain power_domain;
u32 pp;
i915_reg_t pp_ctrl_reg;
wait_panel_off(intel_dp);
/* We got a reference when we enabled the VDD. */
- power_domain = intel_display_port_aux_power_domain(intel_encoder);
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, intel_dp->aux_power_domain);
}
void intel_edp_panel_off(struct intel_dp *intel_dp)
enum port port = dp_to_dig_port(intel_dp)->port;
struct drm_device *dev = encoder->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- enum intel_display_power_domain power_domain;
u32 tmp;
bool ret;
- power_domain = intel_display_port_power_domain(encoder);
- if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
+ if (!intel_display_power_get_if_enabled(dev_priv,
+ encoder->power_domain))
return false;
ret = false;
ret = true;
out:
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, encoder->power_domain);
return ret;
}
if (IS_GEN9_LP(dev_priv))
return DP_TRAIN_VOLTAGE_SWING_LEVEL_3;
else if (INTEL_GEN(dev_priv) >= 9) {
- if (dev_priv->vbt.edp.low_vswing && port == PORT_A)
- return DP_TRAIN_VOLTAGE_SWING_LEVEL_3;
- return DP_TRAIN_VOLTAGE_SWING_LEVEL_2;
+ struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base;
+ return intel_ddi_dp_voltage_max(encoder);
} else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
return DP_TRAIN_VOLTAGE_SWING_LEVEL_3;
else if (IS_GEN7(dev_priv) && port == PORT_A)
static uint8_t intel_dp_autotest_link_training(struct intel_dp *intel_dp)
{
- uint8_t test_result = DP_TEST_ACK;
- return test_result;
+ int status = 0;
+ int min_lane_count = 1;
+ int common_rates[DP_MAX_SUPPORTED_RATES] = {};
+ int link_rate_index, test_link_rate;
+ uint8_t test_lane_count, test_link_bw;
+ /* (DP CTS 1.2)
+ * 4.3.1.11
+ */
+ /* Read the TEST_LANE_COUNT and TEST_LINK_RTAE fields (DP CTS 3.1.4) */
+ status = drm_dp_dpcd_readb(&intel_dp->aux, DP_TEST_LANE_COUNT,
+ &test_lane_count);
+
+ if (status <= 0) {
+ DRM_DEBUG_KMS("Lane count read failed\n");
+ return DP_TEST_NAK;
+ }
+ test_lane_count &= DP_MAX_LANE_COUNT_MASK;
+ /* Validate the requested lane count */
+ if (test_lane_count < min_lane_count ||
+ test_lane_count > intel_dp->max_sink_lane_count)
+ return DP_TEST_NAK;
+
+ status = drm_dp_dpcd_readb(&intel_dp->aux, DP_TEST_LINK_RATE,
+ &test_link_bw);
+ if (status <= 0) {
+ DRM_DEBUG_KMS("Link Rate read failed\n");
+ return DP_TEST_NAK;
+ }
+ /* Validate the requested link rate */
+ test_link_rate = drm_dp_bw_code_to_link_rate(test_link_bw);
+ link_rate_index = intel_dp_link_rate_index(intel_dp,
+ common_rates,
+ test_link_rate);
+ if (link_rate_index < 0)
+ return DP_TEST_NAK;
+
+ intel_dp->compliance.test_lane_count = test_lane_count;
+ intel_dp->compliance.test_link_rate = test_link_rate;
+
+ return DP_TEST_ACK;
}
static uint8_t intel_dp_autotest_video_pattern(struct intel_dp *intel_dp)
{
- uint8_t test_result = DP_TEST_NAK;
- return test_result;
+ uint8_t test_pattern;
+ uint16_t test_misc;
+ __be16 h_width, v_height;
+ int status = 0;
+
+ /* Read the TEST_PATTERN (DP CTS 3.1.5) */
+ status = drm_dp_dpcd_read(&intel_dp->aux, DP_TEST_PATTERN,
+ &test_pattern, 1);
+ if (status <= 0) {
+ DRM_DEBUG_KMS("Test pattern read failed\n");
+ return DP_TEST_NAK;
+ }
+ if (test_pattern != DP_COLOR_RAMP)
+ return DP_TEST_NAK;
+
+ status = drm_dp_dpcd_read(&intel_dp->aux, DP_TEST_H_WIDTH_HI,
+ &h_width, 2);
+ if (status <= 0) {
+ DRM_DEBUG_KMS("H Width read failed\n");
+ return DP_TEST_NAK;
+ }
+
+ status = drm_dp_dpcd_read(&intel_dp->aux, DP_TEST_V_HEIGHT_HI,
+ &v_height, 2);
+ if (status <= 0) {
+ DRM_DEBUG_KMS("V Height read failed\n");
+ return DP_TEST_NAK;
+ }
+
+ status = drm_dp_dpcd_read(&intel_dp->aux, DP_TEST_MISC0,
+ &test_misc, 1);
+ if (status <= 0) {
+ DRM_DEBUG_KMS("TEST MISC read failed\n");
+ return DP_TEST_NAK;
+ }
+ if ((test_misc & DP_TEST_COLOR_FORMAT_MASK) != DP_COLOR_FORMAT_RGB)
+ return DP_TEST_NAK;
+ if (test_misc & DP_TEST_DYNAMIC_RANGE_CEA)
+ return DP_TEST_NAK;
+ switch (test_misc & DP_TEST_BIT_DEPTH_MASK) {
+ case DP_TEST_BIT_DEPTH_6:
+ intel_dp->compliance.test_data.bpc = 6;
+ break;
+ case DP_TEST_BIT_DEPTH_8:
+ intel_dp->compliance.test_data.bpc = 8;
+ break;
+ default:
+ return DP_TEST_NAK;
+ }
+
+ intel_dp->compliance.test_data.video_pattern = test_pattern;
+ intel_dp->compliance.test_data.hdisplay = be16_to_cpu(h_width);
+ intel_dp->compliance.test_data.vdisplay = be16_to_cpu(v_height);
+ /* Set test active flag here so userspace doesn't interrupt things */
+ intel_dp->compliance.test_active = 1;
+
+ return DP_TEST_ACK;
}
static uint8_t intel_dp_autotest_edid(struct intel_dp *intel_dp)
{
- uint8_t test_result = DP_TEST_NAK;
+ uint8_t test_result = DP_TEST_ACK;
struct intel_connector *intel_connector = intel_dp->attached_connector;
struct drm_connector *connector = &intel_connector->base;
DRM_DEBUG_KMS("Failed to write EDID checksum\n");
test_result = DP_TEST_ACK | DP_TEST_EDID_CHECKSUM_WRITE;
- intel_dp->compliance.test_data.edid = INTEL_DP_RESOLUTION_STANDARD;
+ intel_dp->compliance.test_data.edid = INTEL_DP_RESOLUTION_PREFERRED;
}
/* Set test active flag here so userspace doesn't interrupt things */
static void intel_dp_handle_test_request(struct intel_dp *intel_dp)
{
uint8_t response = DP_TEST_NAK;
- uint8_t rxdata = 0;
- int status = 0;
+ uint8_t request = 0;
+ int status;
- status = drm_dp_dpcd_read(&intel_dp->aux, DP_TEST_REQUEST, &rxdata, 1);
+ status = drm_dp_dpcd_readb(&intel_dp->aux, DP_TEST_REQUEST, &request);
if (status <= 0) {
DRM_DEBUG_KMS("Could not read test request from sink\n");
goto update_status;
}
- switch (rxdata) {
+ switch (request) {
case DP_TEST_LINK_TRAINING:
DRM_DEBUG_KMS("LINK_TRAINING test requested\n");
- intel_dp->compliance.test_type = DP_TEST_LINK_TRAINING;
response = intel_dp_autotest_link_training(intel_dp);
break;
case DP_TEST_LINK_VIDEO_PATTERN:
DRM_DEBUG_KMS("TEST_PATTERN test requested\n");
- intel_dp->compliance.test_type = DP_TEST_LINK_VIDEO_PATTERN;
response = intel_dp_autotest_video_pattern(intel_dp);
break;
case DP_TEST_LINK_EDID_READ:
DRM_DEBUG_KMS("EDID test requested\n");
- intel_dp->compliance.test_type = DP_TEST_LINK_EDID_READ;
response = intel_dp_autotest_edid(intel_dp);
break;
case DP_TEST_LINK_PHY_TEST_PATTERN:
DRM_DEBUG_KMS("PHY_PATTERN test requested\n");
- intel_dp->compliance.test_type = DP_TEST_LINK_PHY_TEST_PATTERN;
response = intel_dp_autotest_phy_pattern(intel_dp);
break;
default:
- DRM_DEBUG_KMS("Invalid test request '%02x'\n", rxdata);
+ DRM_DEBUG_KMS("Invalid test request '%02x'\n", request);
break;
}
+ if (response & DP_TEST_ACK)
+ intel_dp->compliance.test_type = request;
+
update_status:
- status = drm_dp_dpcd_write(&intel_dp->aux,
- DP_TEST_RESPONSE,
- &response, 1);
+ status = drm_dp_dpcd_writeb(&intel_dp->aux, DP_TEST_RESPONSE, response);
if (status <= 0)
DRM_DEBUG_KMS("Could not write test response to sink\n");
}
if (!intel_dp->lane_count)
return;
- /* if link training is requested we should perform it always */
- if ((intel_dp->compliance.test_type == DP_TEST_LINK_TRAINING) ||
- (!drm_dp_channel_eq_ok(link_status, intel_dp->lane_count))) {
+ /* Retrain if Channel EQ or CR not ok */
+ if (!drm_dp_channel_eq_ok(link_status, intel_dp->lane_count)) {
DRM_DEBUG_KMS("%s: channel EQ not ok, retraining\n",
intel_encoder->base.name);
intel_dp_short_pulse(struct intel_dp *intel_dp)
{
struct drm_device *dev = intel_dp_to_dev(intel_dp);
+ struct intel_encoder *intel_encoder = &dp_to_dig_port(intel_dp)->base;
u8 sink_irq_vector = 0;
u8 old_sink_count = intel_dp->sink_count;
bool ret;
sink_irq_vector);
if (sink_irq_vector & DP_AUTOMATED_TEST_REQUEST)
- DRM_DEBUG_DRIVER("Test request in short pulse not handled\n");
+ intel_dp_handle_test_request(intel_dp);
if (sink_irq_vector & (DP_CP_IRQ | DP_SINK_SPECIFIC_IRQ))
DRM_DEBUG_DRIVER("CP or sink specific irq unhandled\n");
}
drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
intel_dp_check_link_status(intel_dp);
drm_modeset_unlock(&dev->mode_config.connection_mutex);
+ if (intel_dp->compliance.test_type == DP_TEST_LINK_TRAINING) {
+ DRM_DEBUG_KMS("Link Training Compliance Test requested\n");
+ /* Send a Hotplug Uevent to userspace to start modeset */
+ drm_kms_helper_hotplug_event(intel_encoder->base.dev);
+ }
return true;
}
static enum drm_connector_status
intel_dp_detect_dpcd(struct intel_dp *intel_dp)
{
+ struct intel_lspcon *lspcon = dp_to_lspcon(intel_dp);
uint8_t *dpcd = intel_dp->dpcd;
uint8_t type;
+ if (lspcon->active)
+ lspcon_resume(lspcon);
+
if (!intel_dp_get_dpcd(intel_dp))
return connector_status_disconnected;
struct intel_encoder *intel_encoder = &intel_dig_port->base;
struct drm_device *dev = connector->dev;
enum drm_connector_status status;
- enum intel_display_power_domain power_domain;
u8 sink_irq_vector = 0;
- power_domain = intel_display_port_aux_power_domain(intel_encoder);
- intel_display_power_get(to_i915(dev), power_domain);
+ intel_display_power_get(to_i915(dev), intel_dp->aux_power_domain);
/* Can't disconnect eDP, but you can close the lid... */
if (is_edp(intel_dp))
yesno(intel_dp_source_supports_hbr2(intel_dp)),
yesno(drm_dp_tps3_supported(intel_dp->dpcd)));
- /* Set the max lane count for sink */
- intel_dp->max_sink_lane_count = drm_dp_max_lane_count(intel_dp->dpcd);
+ if (intel_dp->reset_link_params) {
+ /* Set the max lane count for sink */
+ intel_dp->max_sink_lane_count = drm_dp_max_lane_count(intel_dp->dpcd);
+
+ /* Set the max link BW for sink */
+ intel_dp->max_sink_link_bw = intel_dp_max_link_bw(intel_dp);
- /* Set the max link BW for sink */
- intel_dp->max_sink_link_bw = intel_dp_max_link_bw(intel_dp);
+ intel_dp->reset_link_params = false;
+ }
intel_dp_print_rates(intel_dp);
if (status != connector_status_connected && !intel_dp->is_mst)
intel_dp_unset_edid(intel_dp);
- intel_display_power_put(to_i915(dev), power_domain);
+ intel_display_power_put(to_i915(dev), intel_dp->aux_power_domain);
return status;
}
struct intel_dp *intel_dp = intel_attached_dp(connector);
struct intel_encoder *intel_encoder = &dp_to_dig_port(intel_dp)->base;
struct drm_i915_private *dev_priv = to_i915(intel_encoder->base.dev);
- enum intel_display_power_domain power_domain;
DRM_DEBUG_KMS("[CONNECTOR:%d:%s]\n",
connector->base.id, connector->name);
if (connector->status != connector_status_connected)
return;
- power_domain = intel_display_port_aux_power_domain(intel_encoder);
- intel_display_power_get(dev_priv, power_domain);
+ intel_display_power_get(dev_priv, intel_dp->aux_power_domain);
intel_dp_set_edid(intel_dp);
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, intel_dp->aux_power_domain);
if (intel_encoder->type != INTEL_OUTPUT_EDP)
intel_encoder->type = INTEL_OUTPUT_DP;
struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
struct drm_device *dev = intel_dig_port->base.base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- enum intel_display_power_domain power_domain;
lockdep_assert_held(&dev_priv->pps_mutex);
* indefinitely.
*/
DRM_DEBUG_KMS("VDD left on by BIOS, adjusting state tracking\n");
- power_domain = intel_display_port_aux_power_domain(&intel_dig_port->base);
- intel_display_power_get(dev_priv, power_domain);
+ intel_display_power_get(dev_priv, intel_dp->aux_power_domain);
edp_panel_vdd_schedule_off(intel_dp);
}
if (lspcon->active)
lspcon_resume(lspcon);
+ intel_dp->reset_link_params = true;
+
pps_lock(intel_dp);
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
intel_dp_hpd_pulse(struct intel_digital_port *intel_dig_port, bool long_hpd)
{
struct intel_dp *intel_dp = &intel_dig_port->dp;
- struct intel_encoder *intel_encoder = &intel_dig_port->base;
struct drm_device *dev = intel_dig_port->base.base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- enum intel_display_power_domain power_domain;
enum irqreturn ret = IRQ_NONE;
if (intel_dig_port->base.type != INTEL_OUTPUT_EDP &&
long_hpd ? "long" : "short");
if (long_hpd) {
+ intel_dp->reset_link_params = true;
intel_dp->detect_done = false;
return IRQ_NONE;
}
- power_domain = intel_display_port_aux_power_domain(intel_encoder);
- intel_display_power_get(dev_priv, power_domain);
+ intel_display_power_get(dev_priv, intel_dp->aux_power_domain);
if (intel_dp->is_mst) {
if (intel_dp_check_mst_status(intel_dp) == -EINVAL) {
ret = IRQ_HANDLED;
put_power:
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, intel_dp->aux_power_domain);
return ret;
}
return false;
}
+/* Set up the hotplug pin and aux power domain. */
+static void
+intel_dp_init_connector_port_info(struct intel_digital_port *intel_dig_port)
+{
+ struct intel_encoder *encoder = &intel_dig_port->base;
+ struct intel_dp *intel_dp = &intel_dig_port->dp;
+
+ switch (intel_dig_port->port) {
+ case PORT_A:
+ encoder->hpd_pin = HPD_PORT_A;
+ intel_dp->aux_power_domain = POWER_DOMAIN_AUX_A;
+ break;
+ case PORT_B:
+ encoder->hpd_pin = HPD_PORT_B;
+ intel_dp->aux_power_domain = POWER_DOMAIN_AUX_B;
+ break;
+ case PORT_C:
+ encoder->hpd_pin = HPD_PORT_C;
+ intel_dp->aux_power_domain = POWER_DOMAIN_AUX_C;
+ break;
+ case PORT_D:
+ encoder->hpd_pin = HPD_PORT_D;
+ intel_dp->aux_power_domain = POWER_DOMAIN_AUX_D;
+ break;
+ case PORT_E:
+ encoder->hpd_pin = HPD_PORT_E;
+
+ /* FIXME: Check VBT for actual wiring of PORT E */
+ intel_dp->aux_power_domain = POWER_DOMAIN_AUX_D;
+ break;
+ default:
+ MISSING_CASE(intel_dig_port->port);
+ }
+}
+
bool
intel_dp_init_connector(struct intel_digital_port *intel_dig_port,
struct intel_connector *intel_connector)
intel_dig_port->max_lanes, port_name(port)))
return false;
+ intel_dp->reset_link_params = true;
intel_dp->pps_pipe = INVALID_PIPE;
intel_dp->active_pipe = INVALID_PIPE;
connector->interlace_allowed = true;
connector->doublescan_allowed = 0;
+ intel_dp_init_connector_port_info(intel_dig_port);
+
intel_dp_aux_init(intel_dp);
INIT_DELAYED_WORK(&intel_dp->panel_vdd_work,
else
intel_connector->get_hw_state = intel_connector_get_hw_state;
- /* Set up the hotplug pin. */
- switch (port) {
- case PORT_A:
- intel_encoder->hpd_pin = HPD_PORT_A;
- break;
- case PORT_B:
- intel_encoder->hpd_pin = HPD_PORT_B;
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1))
- intel_encoder->hpd_pin = HPD_PORT_A;
- break;
- case PORT_C:
- intel_encoder->hpd_pin = HPD_PORT_C;
- break;
- case PORT_D:
- intel_encoder->hpd_pin = HPD_PORT_D;
- break;
- case PORT_E:
- intel_encoder->hpd_pin = HPD_PORT_E;
- break;
- default:
- BUG();
- }
-
/* init MST on ports that can support it */
if (HAS_DP_MST(dev_priv) && !is_edp(intel_dp) &&
(port == PORT_B || port == PORT_C || port == PORT_D))
intel_dig_port->max_lanes = 4;
intel_encoder->type = INTEL_OUTPUT_DP;
+ intel_encoder->power_domain = intel_port_to_power_domain(port);
if (IS_CHERRYVIEW(dev_priv)) {
if (port == PORT_D)
intel_encoder->crtc_mask = 1 << 2;
pipe_config->has_pch_encoder = false;
bpp = 24;
+ if (intel_dp->compliance.test_data.bpc) {
+ bpp = intel_dp->compliance.test_data.bpc * 3;
+ DRM_DEBUG_KMS("Setting pipe bpp to %d\n",
+ bpp);
+ }
/*
* for MST we always configure max link bw - the spec doesn't
* seem to suggest we should do otherwise.
pipe_config->lane_count = lane_count;
- pipe_config->pipe_bpp = 24;
+ pipe_config->pipe_bpp = bpp;
pipe_config->port_clock = intel_dp_max_link_rate(intel_dp);
state = pipe_config->base.state;
struct intel_dp *intel_dp = &intel_dig_port->dp;
struct intel_connector *connector =
to_intel_connector(old_conn_state->connector);
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
int ret;
DRM_DEBUG_KMS("%d\n", intel_dp->active_mst_links);
if (ret) {
DRM_ERROR("failed to update payload %d\n", ret);
}
- if (old_crtc_state->has_audio) {
+ if (old_crtc_state->has_audio)
intel_audio_codec_disable(encoder);
- intel_display_power_put(dev_priv, POWER_DOMAIN_AUDIO);
- }
}
static void intel_mst_post_disable_dp(struct intel_encoder *encoder,
DRM_DEBUG_KMS("%d\n", intel_dp->active_mst_links);
- if (intel_dp->active_mst_links == 0) {
- intel_ddi_clk_select(&intel_dig_port->base,
- pipe_config->shared_dpll);
-
- intel_prepare_dp_ddi_buffers(&intel_dig_port->base);
- intel_dp_set_link_params(intel_dp,
- pipe_config->port_clock,
- pipe_config->lane_count,
- true);
-
- intel_ddi_init_dp_buf_reg(&intel_dig_port->base);
-
- intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_ON);
-
- intel_dp_start_link_train(intel_dp);
- intel_dp_stop_link_train(intel_dp);
- }
+ if (intel_dp->active_mst_links == 0)
+ intel_dig_port->base.pre_enable(&intel_dig_port->base,
+ pipe_config, NULL);
ret = drm_dp_mst_allocate_vcpi(&intel_dp->mst_mgr,
connector->port,
ret = drm_dp_check_act_status(&intel_dp->mst_mgr);
ret = drm_dp_update_payload_part2(&intel_dp->mst_mgr);
- if (pipe_config->has_audio) {
- intel_display_power_get(dev_priv, POWER_DOMAIN_AUDIO);
+ if (pipe_config->has_audio)
intel_audio_codec_enable(encoder, pipe_config, conn_state);
- }
}
static bool intel_dp_mst_enc_get_hw_state(struct intel_encoder *encoder,
DRM_MODE_ENCODER_DPMST, "DP-MST %c", pipe_name(pipe));
intel_encoder->type = INTEL_OUTPUT_DP_MST;
+ intel_encoder->power_domain = intel_dig_port->base.power_domain;
intel_encoder->port = intel_dig_port->port;
intel_encoder->crtc_mask = 0x7;
intel_encoder->cloneable = 0;
* commit phase.
*/
-struct intel_shared_dpll *
-skl_find_link_pll(struct drm_i915_private *dev_priv, int clock)
-{
- struct intel_shared_dpll *pll = NULL;
- struct intel_dpll_hw_state dpll_hw_state;
- enum intel_dpll_id i;
- bool found = false;
-
- if (!skl_ddi_dp_set_dpll_hw_state(clock, &dpll_hw_state))
- return pll;
-
- for (i = DPLL_ID_SKL_DPLL1; i <= DPLL_ID_SKL_DPLL3; i++) {
- pll = &dev_priv->shared_dplls[i];
-
- /* Only want to check enabled timings first */
- if (pll->state.crtc_mask == 0)
- continue;
-
- if (memcmp(&dpll_hw_state, &pll->state.hw_state,
- sizeof(pll->state.hw_state)) == 0) {
- found = true;
- break;
- }
- }
-
- /* Ok no matching timings, maybe there's a free one? */
- for (i = DPLL_ID_SKL_DPLL1;
- ((found == false) && (i <= DPLL_ID_SKL_DPLL3)); i++) {
- pll = &dev_priv->shared_dplls[i];
- if (pll->state.crtc_mask == 0) {
- pll->state.hw_state = dpll_hw_state;
- break;
- }
- }
-
- return pll;
-}
-
static void
intel_atomic_duplicate_dpll_state(struct drm_i915_private *dev_priv,
struct intel_shared_dpll_state *shared_dpll)
return pll;
}
-struct intel_shared_dpll *hsw_ddi_dp_get_dpll(struct intel_encoder *encoder,
- int clock)
+static struct intel_shared_dpll *
+hsw_ddi_dp_get_dpll(struct intel_encoder *encoder, int clock)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_shared_dpll *pll;
}
-bool skl_ddi_dp_set_dpll_hw_state(int clock,
- struct intel_dpll_hw_state *dpll_hw_state)
+static bool
+skl_ddi_dp_set_dpll_hw_state(int clock,
+ struct intel_dpll_hw_state *dpll_hw_state)
{
uint32_t ctrl1;
return true;
}
-bool bxt_ddi_dp_set_dpll_hw_state(int clock,
- struct intel_dpll_hw_state *dpll_hw_state)
+static bool
+bxt_ddi_dp_set_dpll_hw_state(int clock,
+ struct intel_dpll_hw_state *dpll_hw_state)
{
struct bxt_clk_div clk_div = {0};
const struct dpll_info *dpll_info;
int i;
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ if (IS_GEN9_BC(dev_priv))
dpll_mgr = &skl_pll_mgr;
else if (IS_GEN9_LP(dev_priv))
dpll_mgr = &bxt_pll_mgr;
void intel_dpll_dump_hw_state(struct drm_i915_private *dev_priv,
struct intel_dpll_hw_state *hw_state);
-/* BXT dpll related functions */
-bool bxt_ddi_dp_set_dpll_hw_state(int clock,
- struct intel_dpll_hw_state *dpll_hw_state);
-
-
-/* SKL dpll related functions */
-bool skl_ddi_dp_set_dpll_hw_state(int clock,
- struct intel_dpll_hw_state *dpll_hw_state);
-struct intel_shared_dpll *skl_find_link_pll(struct drm_i915_private *dev_priv,
- int clock);
-
-
-/* HSW dpll related functions */
-struct intel_shared_dpll *hsw_ddi_dp_get_dpll(struct intel_encoder *encoder,
- int clock);
-
#endif /* _INTEL_DPLL_MGR_H_ */
* be set correctly before calling this function. */
void (*get_config)(struct intel_encoder *,
struct intel_crtc_state *pipe_config);
+ /* Returns a mask of power domains that need to be referenced as part
+ * of the hardware state readout code. */
+ u64 (*get_power_domains)(struct intel_encoder *encoder);
/*
* Called during system suspend after all pending requests for the
* encoder are flushed (for example for DP AUX transactions) and
void (*suspend)(struct intel_encoder *);
int crtc_mask;
enum hpd_pin hpd_pin;
+ enum intel_display_power_domain power_domain;
/* for communication with audio component; protected by av_mutex */
const struct drm_connector *audio_connector;
};
struct intel_atomic_state {
struct drm_atomic_state base;
- unsigned int cdclk;
-
- /*
- * Calculated device cdclk, can be different from cdclk
- * only when all crtc's are DPMS off.
- */
- unsigned int dev_cdclk;
+ struct {
+ /*
+ * Logical state of cdclk (used for all scaling, watermark,
+ * etc. calculations and checks). This is computed as if all
+ * enabled crtcs were active.
+ */
+ struct intel_cdclk_state logical;
+
+ /*
+ * Actual state of cdclk, can be different from the logical
+ * state only when all crtc's are DPMS off.
+ */
+ struct intel_cdclk_state actual;
+ } cdclk;
bool dpll_set, modeset;
unsigned int active_crtcs;
unsigned int min_pixclk[I915_MAX_PIPES];
- /* SKL/KBL Only */
- unsigned int cdclk_pll_vco;
-
struct intel_shared_dpll_state shared_dpll[I915_NUM_PLLS];
/*
uint32_t linetime;
};
+enum vlv_wm_level {
+ VLV_WM_LEVEL_PM2,
+ VLV_WM_LEVEL_PM5,
+ VLV_WM_LEVEL_DDR_DVFS,
+ NUM_VLV_WM_LEVELS,
+};
+
+struct vlv_wm_state {
+ struct vlv_pipe_wm wm[NUM_VLV_WM_LEVELS];
+ struct vlv_sr_wm sr[NUM_VLV_WM_LEVELS];
+ uint8_t num_levels;
+ bool cxsr;
+};
+
+struct vlv_fifo_state {
+ u16 plane[I915_MAX_PLANES];
+};
+
struct intel_crtc_wm_state {
union {
struct {
struct skl_pipe_wm optimal;
struct skl_ddb_entry ddb;
} skl;
+
+ struct {
+ /* "raw" watermarks (not inverted) */
+ struct vlv_pipe_wm raw[NUM_VLV_WM_LEVELS];
+ /* intermediate watermarks (inverted) */
+ struct vlv_wm_state intermediate;
+ /* optimal watermarks (inverted) */
+ struct vlv_wm_state optimal;
+ /* display FIFO split */
+ struct vlv_fifo_state fifo_state;
+ } vlv;
};
/*
bool disable_cxsr;
bool update_wm_pre, update_wm_post; /* watermarks are updated */
bool fb_changed; /* fb on any of the planes is changed */
+ bool fifo_changed; /* FIFO split is changed */
/* Pipe source size (ie. panel fitter input size)
* All planes will be positioned inside this space,
* and get clipped at the edges. */
int pipe_src_w, pipe_src_h;
+ /*
+ * Pipe pixel rate, adjusted for
+ * panel fitter/pipe scaler downscaling.
+ */
+ unsigned int pixel_rate;
+
/* Whether to set up the PCH/FDI. Note that we never allow sharing
* between pch encoders and cpu encoders. */
bool has_pch_encoder;
*/
bool dither;
+ /*
+ * Dither gets enabled for 18bpp which causes CRC mismatch errors for
+ * compliance video pattern tests.
+ * Disable dither only if it is a compliance test request for
+ * 18bpp.
+ */
+ bool dither_force_disable;
+
/* Controls for the clock computation, to override various stages. */
bool clock_set;
/* Gamma mode programmed on the pipe */
uint32_t gamma_mode;
-};
-struct vlv_wm_state {
- struct vlv_pipe_wm wm[3];
- struct vlv_sr_wm sr[3];
- uint8_t num_active_planes;
- uint8_t num_levels;
- uint8_t level;
- bool cxsr;
+ /* bitmask of visible planes (enum plane_id) */
+ u8 active_planes;
};
struct intel_crtc {
bool active;
bool lowfreq_avail;
u8 plane_ids_mask;
- unsigned long enabled_power_domains;
+ unsigned long long enabled_power_domains;
struct intel_overlay *overlay;
struct intel_flip_work *flip_work;
/* watermarks currently being used */
union {
struct intel_pipe_wm ilk;
+ struct vlv_wm_state vlv;
} active;
-
- /* allow CxSR on this pipe */
- bool cxsr_allowed;
} wm;
int scanline_offset;
/* scalers available on this crtc */
int num_scalers;
-
- struct vlv_wm_state wm_state;
-};
-
-struct intel_plane_wm_parameters {
- uint32_t horiz_pixels;
- uint32_t vert_pixels;
- /*
- * For packed pixel formats:
- * bytes_per_pixel - holds bytes per pixel
- * For planar pixel formats:
- * bytes_per_pixel - holds bytes per pixel for uv-plane
- * y_bytes_per_pixel - holds bytes per pixel for y-plane
- */
- uint8_t bytes_per_pixel;
- uint8_t y_bytes_per_pixel;
- bool enabled;
- bool scaled;
- u64 tiling;
- unsigned int rotation;
- uint16_t fifo_size;
};
struct intel_plane {
int max_downscale;
uint32_t frontbuffer_bit;
- /* Since we need to change the watermarks before/after
- * enabling/disabling the planes, we need to store the parameters here
- * as the other pieces of the struct may not reflect the values we want
- * for the watermark calculations. Currently only Haswell uses this.
- */
- struct intel_plane_wm_parameters wm;
-
/*
* NOTE: Do not place new plane state fields here (e.g., when adding
* new plane properties). New runtime state should now be placed in
struct intel_dp_compliance_data {
unsigned long edid;
+ uint8_t video_pattern;
+ uint16_t hdisplay, vdisplay;
+ uint8_t bpc;
};
struct intel_dp_compliance {
unsigned long test_type;
struct intel_dp_compliance_data test_data;
bool test_active;
+ int test_link_rate;
+ u8 test_lane_count;
};
struct intel_dp {
bool has_audio;
bool detect_done;
bool channel_eq_status;
+ bool reset_link_params;
enum hdmi_force_audio force_audio;
bool limited_color_range;
bool color_range_auto;
/* sink or branch descriptor */
struct intel_dp_desc desc;
struct drm_dp_aux aux;
+ enum intel_display_power_domain aux_power_domain;
uint8_t train_set[4];
int panel_power_up_delay;
int panel_power_down_delay;
struct intel_lspcon {
bool active;
enum drm_lspcon_mode mode;
- bool desc_valid;
};
struct intel_digital_port {
enum irqreturn (*hpd_pulse)(struct intel_digital_port *, bool);
bool release_cl2_override;
uint8_t max_lanes;
+ enum intel_display_power_domain ddi_io_power_domain;
};
struct intel_dp_mst_encoder {
static inline struct intel_digital_port *
enc_to_dig_port(struct drm_encoder *encoder)
{
- return container_of(encoder, struct intel_digital_port, base.base);
+ struct intel_encoder *intel_encoder = to_intel_encoder(encoder);
+
+ switch (intel_encoder->type) {
+ case INTEL_OUTPUT_UNKNOWN:
+ WARN_ON(!HAS_DDI(to_i915(encoder->dev)));
+ case INTEL_OUTPUT_DP:
+ case INTEL_OUTPUT_EDP:
+ case INTEL_OUTPUT_HDMI:
+ return container_of(encoder, struct intel_digital_port,
+ base.base);
+ default:
+ return NULL;
+ }
}
static inline struct intel_dp_mst_encoder *
void gen6_reset_rps_interrupts(struct drm_i915_private *dev_priv);
void gen6_enable_rps_interrupts(struct drm_i915_private *dev_priv);
void gen6_disable_rps_interrupts(struct drm_i915_private *dev_priv);
-u32 gen6_sanitize_rps_pm_mask(struct drm_i915_private *dev_priv, u32 mask);
+
+static inline u32 gen6_sanitize_rps_pm_mask(const struct drm_i915_private *i915,
+ u32 mask)
+{
+ return mask & ~i915->rps.pm_intrmsk_mbz;
+}
+
void intel_runtime_pm_disable_interrupts(struct drm_i915_private *dev_priv);
void intel_runtime_pm_enable_interrupts(struct drm_i915_private *dev_priv);
static inline bool intel_irqs_enabled(struct drm_i915_private *dev_priv)
struct intel_crtc_state *old_crtc_state,
struct drm_connector_state *old_conn_state);
void intel_prepare_dp_ddi_buffers(struct intel_encoder *encoder);
-void hsw_fdi_link_train(struct drm_crtc *crtc);
+void hsw_fdi_link_train(struct intel_crtc *crtc,
+ const struct intel_crtc_state *crtc_state);
void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port);
enum port intel_ddi_get_encoder_port(struct intel_encoder *intel_encoder);
bool intel_ddi_get_hw_state(struct intel_encoder *encoder, enum pipe *pipe);
-void intel_ddi_enable_transcoder_func(struct drm_crtc *crtc);
+void intel_ddi_enable_transcoder_func(const struct intel_crtc_state *crtc_state);
void intel_ddi_disable_transcoder_func(struct drm_i915_private *dev_priv,
enum transcoder cpu_transcoder);
-void intel_ddi_enable_pipe_clock(struct intel_crtc *intel_crtc);
-void intel_ddi_disable_pipe_clock(struct intel_crtc *intel_crtc);
+void intel_ddi_enable_pipe_clock(const struct intel_crtc_state *crtc_state);
+void intel_ddi_disable_pipe_clock(const struct intel_crtc_state *crtc_state);
bool intel_ddi_pll_select(struct intel_crtc *crtc,
struct intel_crtc_state *crtc_state);
-void intel_ddi_set_pipe_settings(struct drm_crtc *crtc);
+void intel_ddi_set_pipe_settings(const struct intel_crtc_state *crtc_state);
void intel_ddi_prepare_link_retrain(struct intel_dp *intel_dp);
bool intel_ddi_connector_get_hw_state(struct intel_connector *intel_connector);
bool intel_ddi_is_audio_enabled(struct drm_i915_private *dev_priv,
struct intel_crtc *intel_crtc);
void intel_ddi_get_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config);
-struct intel_encoder *
-intel_ddi_get_crtc_new_encoder(struct intel_crtc_state *crtc_state);
void intel_ddi_init_dp_buf_reg(struct intel_encoder *encoder);
void intel_ddi_clock_get(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config);
-void intel_ddi_set_vc_payload_alloc(struct drm_crtc *crtc, bool state);
+void intel_ddi_set_vc_payload_alloc(const struct intel_crtc_state *crtc_state,
+ bool state);
uint32_t ddi_signal_levels(struct intel_dp *intel_dp);
-struct intel_shared_dpll *intel_ddi_get_link_dpll(struct intel_dp *intel_dp,
- int clock);
-unsigned int intel_fb_align_height(struct drm_device *dev,
- unsigned int height,
- uint32_t pixel_format,
- uint64_t fb_format_modifier);
-u32 intel_fb_stride_alignment(const struct drm_i915_private *dev_priv,
- uint64_t fb_modifier, uint32_t pixel_format);
+u8 intel_ddi_dp_voltage_max(struct intel_encoder *encoder);
+
+unsigned int intel_fb_align_height(const struct drm_framebuffer *fb,
+ int plane, unsigned int height);
/* intel_audio.c */
void intel_init_audio_hooks(struct drm_i915_private *dev_priv);
void intel_audio_init(struct drm_i915_private *dev_priv);
void intel_audio_deinit(struct drm_i915_private *dev_priv);
+/* intel_cdclk.c */
+void skl_init_cdclk(struct drm_i915_private *dev_priv);
+void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
+void bxt_init_cdclk(struct drm_i915_private *dev_priv);
+void bxt_uninit_cdclk(struct drm_i915_private *dev_priv);
+void intel_init_cdclk_hooks(struct drm_i915_private *dev_priv);
+void intel_update_max_cdclk(struct drm_i915_private *dev_priv);
+void intel_update_cdclk(struct drm_i915_private *dev_priv);
+void intel_update_rawclk(struct drm_i915_private *dev_priv);
+bool intel_cdclk_state_compare(const struct intel_cdclk_state *a,
+ const struct intel_cdclk_state *b);
+void intel_set_cdclk(struct drm_i915_private *dev_priv,
+ const struct intel_cdclk_state *cdclk_state);
+
/* intel_display.c */
enum transcoder intel_crtc_pch_transcoder(struct intel_crtc *crtc);
-void skl_set_preferred_cdclk_vco(struct drm_i915_private *dev_priv, int vco);
void intel_update_rawclk(struct drm_i915_private *dev_priv);
+int vlv_get_hpll_vco(struct drm_i915_private *dev_priv);
int vlv_get_cck_clock(struct drm_i915_private *dev_priv,
const char *name, u32 reg, int ref_freq);
+int vlv_get_cck_clock_hpll(struct drm_i915_private *dev_priv,
+ const char *name, u32 reg);
void lpt_disable_pch_transcoder(struct drm_i915_private *dev_priv);
void lpt_disable_iclkip(struct drm_i915_private *dev_priv);
extern const struct drm_plane_funcs intel_plane_funcs;
intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb, unsigned int rotation);
void intel_unpin_fb_vma(struct i915_vma *vma);
struct drm_framebuffer *
-__intel_framebuffer_create(struct drm_device *dev,
- struct drm_mode_fb_cmd2 *mode_cmd,
- struct drm_i915_gem_object *obj);
+intel_framebuffer_create(struct drm_i915_gem_object *obj,
+ struct drm_mode_fb_cmd2 *mode_cmd);
void intel_finish_page_flip_cs(struct drm_i915_private *dev_priv, int pipe);
void intel_finish_page_flip_mmio(struct drm_i915_private *dev_priv, int pipe);
void intel_check_page_flip(struct drm_i915_private *dev_priv, int pipe);
int intel_plane_atomic_calc_changes(struct drm_crtc_state *crtc_state,
struct drm_plane_state *plane_state);
-unsigned int intel_tile_height(const struct drm_i915_private *dev_priv,
- uint64_t fb_modifier, unsigned int cpp);
-
void assert_pch_transcoder_disabled(struct drm_i915_private *dev_priv,
enum pipe pipe);
void intel_finish_reset(struct drm_i915_private *dev_priv);
void hsw_enable_pc8(struct drm_i915_private *dev_priv);
void hsw_disable_pc8(struct drm_i915_private *dev_priv);
-void bxt_init_cdclk(struct drm_i915_private *dev_priv);
-void bxt_uninit_cdclk(struct drm_i915_private *dev_priv);
void gen9_sanitize_dc_state(struct drm_i915_private *dev_priv);
void bxt_enable_dc9(struct drm_i915_private *dev_priv);
void bxt_disable_dc9(struct drm_i915_private *dev_priv);
void gen9_enable_dc5(struct drm_i915_private *dev_priv);
-void skl_init_cdclk(struct drm_i915_private *dev_priv);
-void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
unsigned int skl_cdclk_get_vco(unsigned int freq);
void skl_enable_dc6(struct drm_i915_private *dev_priv);
void skl_disable_dc6(struct drm_i915_private *dev_priv);
bool intel_crtc_active(struct intel_crtc *crtc);
void hsw_enable_ips(struct intel_crtc *crtc);
void hsw_disable_ips(struct intel_crtc *crtc);
-enum intel_display_power_domain
-intel_display_port_power_domain(struct intel_encoder *intel_encoder);
-enum intel_display_power_domain
-intel_display_port_aux_power_domain(struct intel_encoder *intel_encoder);
+enum intel_display_power_domain intel_port_to_power_domain(enum port port);
void intel_mode_from_pipe_config(struct drm_display_mode *mode,
struct intel_crtc_state *pipe_config);
void intel_power_domains_fini(struct drm_i915_private *);
void intel_power_domains_init_hw(struct drm_i915_private *dev_priv, bool resume);
void intel_power_domains_suspend(struct drm_i915_private *dev_priv);
+void intel_power_domains_verify_state(struct drm_i915_private *dev_priv);
void bxt_display_core_init(struct drm_i915_private *dev_priv, bool resume);
void bxt_display_core_uninit(struct drm_i915_private *dev_priv);
void intel_runtime_pm_enable(struct drm_i915_private *dev_priv);
assert_rpm_wakelock_held(struct drm_i915_private *dev_priv)
{
assert_rpm_device_not_suspended(dev_priv);
- /* FIXME: Needs to be converted back to WARN_ONCE, but currently causes
- * too much noise. */
- if (!atomic_read(&dev_priv->pm.wakeref_count))
- DRM_DEBUG_DRIVER("RPM wakelock ref not held during HW access");
+ WARN_ONCE(!atomic_read(&dev_priv->pm.wakeref_count),
+ "RPM wakelock ref not held during HW access");
}
/**
struct skl_ddb_allocation *ddb /* out */);
void skl_pipe_wm_get_hw_state(struct drm_crtc *crtc,
struct skl_pipe_wm *out);
+void vlv_wm_sanitize(struct drm_i915_private *dev_priv);
bool intel_can_enable_sagv(struct drm_atomic_state *state);
int intel_enable_sagv(struct drm_i915_private *dev_priv);
int intel_disable_sagv(struct drm_i915_private *dev_priv);
bool skl_ddb_allocation_overlaps(const struct skl_ddb_entry **entries,
const struct skl_ddb_entry *ddb,
int ignore);
-uint32_t ilk_pipe_pixel_rate(const struct intel_crtc_state *pipe_config);
bool ilk_disable_lp_wm(struct drm_device *dev);
int sanitize_rc6_option(struct drm_i915_private *dev_priv, int enable_rc6);
static inline int intel_enable_rc6(void)
return to_intel_plane_state(plane_state);
}
-int intel_atomic_setup_scalers(struct drm_device *dev,
- struct intel_crtc *intel_crtc,
- struct intel_crtc_state *crtc_state);
+int intel_atomic_setup_scalers(struct drm_i915_private *dev_priv,
+ struct intel_crtc *intel_crtc,
+ struct intel_crtc_state *crtc_state);
/* intel_atomic_plane.c */
struct intel_plane_state *intel_create_plane_state(struct drm_plane *plane);
/* intel_pipe_crc.c */
int intel_pipe_crc_create(struct drm_minor *minor);
-void intel_pipe_crc_cleanup(struct drm_minor *minor);
#ifdef CONFIG_DEBUG_FS
int intel_crtc_set_crc_source(struct drm_crtc *crtc, const char *source_name,
size_t *values_cnt);
#include <drm/drm_crtc.h>
#include <drm/drm_edid.h>
#include <drm/i915_drm.h>
-#include <drm/drm_panel.h>
#include <drm/drm_mipi_dsi.h>
#include <linux/slab.h>
#include <linux/gpio/consumer.h>
#include "intel_drv.h"
#include "intel_dsi.h"
-static const struct {
- u16 panel_id;
- struct drm_panel * (*init)(struct intel_dsi *intel_dsi, u16 panel_id);
-} intel_dsi_drivers[] = {
- {
- .panel_id = MIPI_DSI_GENERIC_PANEL_ID,
- .init = vbt_panel_init,
- },
-};
-
/* return pixels in terms of txbyteclkhs */
static u16 txbyteclkhs(u16 pixels, int bpp, int lane_count,
u16 burst_mode_ratio)
}
}
-static void wait_for_dsi_fifo_empty(struct intel_dsi *intel_dsi, enum port port)
+void wait_for_dsi_fifo_empty(struct intel_dsi *intel_dsi, enum port port)
{
struct drm_encoder *encoder = &intel_dsi->base.base;
struct drm_device *dev = encoder->dev;
return true;
}
-static void bxt_dsi_device_ready(struct intel_encoder *encoder)
+static void glk_dsi_device_ready(struct intel_encoder *encoder)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
enum port port;
- u32 val;
+ u32 tmp, val;
- DRM_DEBUG_KMS("\n");
+ /* Set the MIPI mode
+ * If MIPI_Mode is off, then writing to LP_Wake bit is not reflecting.
+ * Power ON MIPI IO first and then write into IO reset and LP wake bits
+ */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ tmp = I915_READ(MIPI_CTRL(port));
+ I915_WRITE(MIPI_CTRL(port), tmp | GLK_MIPIIO_ENABLE);
+ }
+
+ /* Put the IO into reset */
+ tmp = I915_READ(MIPI_CTRL(PORT_A));
+ tmp &= ~GLK_MIPIIO_RESET_RELEASED;
+ I915_WRITE(MIPI_CTRL(PORT_A), tmp);
+
+ /* Program LP Wake */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ tmp = I915_READ(MIPI_CTRL(port));
+ tmp |= GLK_LP_WAKE;
+ I915_WRITE(MIPI_CTRL(port), tmp);
+ }
- /* Exit Low power state in 4 steps*/
+ /* Wait for Pwr ACK */
for_each_dsi_port(port, intel_dsi->ports) {
+ if (intel_wait_for_register(dev_priv,
+ MIPI_CTRL(port), GLK_MIPIIO_PORT_POWERED,
+ GLK_MIPIIO_PORT_POWERED, 20))
+ DRM_ERROR("MIPIO port is powergated\n");
+ }
- /* 1. Enable MIPI PHY transparent latch */
- val = I915_READ(BXT_MIPI_PORT_CTRL(port));
- I915_WRITE(BXT_MIPI_PORT_CTRL(port), val | LP_OUTPUT_HOLD);
- usleep_range(2000, 2500);
+ /* Wait for MIPI PHY status bit to set */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ if (intel_wait_for_register(dev_priv,
+ MIPI_CTRL(port), GLK_PHY_STATUS_PORT_READY,
+ GLK_PHY_STATUS_PORT_READY, 20))
+ DRM_ERROR("PHY is not ON\n");
+ }
- /* 2. Enter ULPS */
+ /* Get IO out of reset */
+ tmp = I915_READ(MIPI_CTRL(PORT_A));
+ I915_WRITE(MIPI_CTRL(PORT_A), tmp | GLK_MIPIIO_RESET_RELEASED);
+
+ /* Get IO out of Low power state*/
+ for_each_dsi_port(port, intel_dsi->ports) {
+ if (!(I915_READ(MIPI_DEVICE_READY(port)) & DEVICE_READY)) {
+ val = I915_READ(MIPI_DEVICE_READY(port));
+ val &= ~ULPS_STATE_MASK;
+ val |= DEVICE_READY;
+ I915_WRITE(MIPI_DEVICE_READY(port), val);
+ usleep_range(10, 15);
+ }
+
+ /* Enter ULPS */
val = I915_READ(MIPI_DEVICE_READY(port));
val &= ~ULPS_STATE_MASK;
val |= (ULPS_STATE_ENTER | DEVICE_READY);
I915_WRITE(MIPI_DEVICE_READY(port), val);
- /* at least 2us - relaxed for hrtimer subsystem optimization */
- usleep_range(10, 50);
- /* 3. Exit ULPS */
+ /* Wait for ULPS Not active */
+ if (intel_wait_for_register(dev_priv,
+ MIPI_CTRL(port), GLK_ULPS_NOT_ACTIVE,
+ GLK_ULPS_NOT_ACTIVE, 20))
+ DRM_ERROR("ULPS is still active\n");
+
+ /* Exit ULPS */
val = I915_READ(MIPI_DEVICE_READY(port));
val &= ~ULPS_STATE_MASK;
val |= (ULPS_STATE_EXIT | DEVICE_READY);
I915_WRITE(MIPI_DEVICE_READY(port), val);
- usleep_range(1000, 1500);
- /* Clear ULPS and set device ready */
+ /* Enter Normal Mode */
+ val = I915_READ(MIPI_DEVICE_READY(port));
+ val &= ~ULPS_STATE_MASK;
+ val |= (ULPS_STATE_NORMAL_OPERATION | DEVICE_READY);
+ I915_WRITE(MIPI_DEVICE_READY(port), val);
+
+ tmp = I915_READ(MIPI_CTRL(port));
+ tmp &= ~GLK_LP_WAKE;
+ I915_WRITE(MIPI_CTRL(port), tmp);
+ }
+
+ /* Wait for Stop state */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ if (intel_wait_for_register(dev_priv,
+ MIPI_CTRL(port), GLK_DATA_LANE_STOP_STATE,
+ GLK_DATA_LANE_STOP_STATE, 20))
+ DRM_ERROR("Date lane not in STOP state\n");
+ }
+
+ /* Wait for AFE LATCH */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ if (intel_wait_for_register(dev_priv,
+ BXT_MIPI_PORT_CTRL(port), AFE_LATCHOUT,
+ AFE_LATCHOUT, 20))
+ DRM_ERROR("D-PHY not entering LP-11 state\n");
+ }
+}
+
+static void bxt_dsi_device_ready(struct intel_encoder *encoder)
+{
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
+ enum port port;
+ u32 val;
+
+ DRM_DEBUG_KMS("\n");
+
+ /* Enable MIPI PHY transparent latch */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ val = I915_READ(BXT_MIPI_PORT_CTRL(port));
+ I915_WRITE(BXT_MIPI_PORT_CTRL(port), val | LP_OUTPUT_HOLD);
+ usleep_range(2000, 2500);
+ }
+
+ /* Clear ULPS and set device ready */
+ for_each_dsi_port(port, intel_dsi->ports) {
val = I915_READ(MIPI_DEVICE_READY(port));
val &= ~ULPS_STATE_MASK;
+ I915_WRITE(MIPI_DEVICE_READY(port), val);
+ usleep_range(2000, 2500);
val |= DEVICE_READY;
I915_WRITE(MIPI_DEVICE_READY(port), val);
}
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
vlv_dsi_device_ready(encoder);
- else if (IS_GEN9_LP(dev_priv))
+ else if (IS_BROXTON(dev_priv))
bxt_dsi_device_ready(encoder);
+ else if (IS_GEMINILAKE(dev_priv))
+ glk_dsi_device_ready(encoder);
+}
+
+static void glk_dsi_enter_low_power_mode(struct intel_encoder *encoder)
+{
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
+ enum port port;
+ u32 val;
+
+ /* Enter ULPS */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ val = I915_READ(MIPI_DEVICE_READY(port));
+ val &= ~ULPS_STATE_MASK;
+ val |= (ULPS_STATE_ENTER | DEVICE_READY);
+ I915_WRITE(MIPI_DEVICE_READY(port), val);
+ }
+
+ /* Wait for MIPI PHY status bit to unset */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ if (intel_wait_for_register(dev_priv,
+ MIPI_CTRL(port),
+ GLK_PHY_STATUS_PORT_READY, 0, 20))
+ DRM_ERROR("PHY is not turning OFF\n");
+ }
+
+ /* Wait for Pwr ACK bit to unset */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ if (intel_wait_for_register(dev_priv,
+ MIPI_CTRL(port),
+ GLK_MIPIIO_PORT_POWERED, 0, 20))
+ DRM_ERROR("MIPI IO Port is not powergated\n");
+ }
+}
+
+static void glk_dsi_disable_mipi_io(struct intel_encoder *encoder)
+{
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
+ enum port port;
+ u32 tmp;
+
+ /* Put the IO into reset */
+ tmp = I915_READ(MIPI_CTRL(PORT_A));
+ tmp &= ~GLK_MIPIIO_RESET_RELEASED;
+ I915_WRITE(MIPI_CTRL(PORT_A), tmp);
+
+ /* Wait for MIPI PHY status bit to unset */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ if (intel_wait_for_register(dev_priv,
+ MIPI_CTRL(port),
+ GLK_PHY_STATUS_PORT_READY, 0, 20))
+ DRM_ERROR("PHY is not turning OFF\n");
+ }
+
+ /* Clear MIPI mode */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ tmp = I915_READ(MIPI_CTRL(port));
+ tmp &= ~GLK_MIPIIO_ENABLE;
+ I915_WRITE(MIPI_CTRL(port), tmp);
+ }
+}
+
+static void glk_dsi_clear_device_ready(struct intel_encoder *encoder)
+{
+ glk_dsi_enter_low_power_mode(encoder);
+ glk_dsi_disable_mipi_io(encoder);
+}
+
+static void vlv_dsi_clear_device_ready(struct intel_encoder *encoder)
+{
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
+ enum port port;
+
+ DRM_DEBUG_KMS("\n");
+ for_each_dsi_port(port, intel_dsi->ports) {
+ /* Common bit for both MIPI Port A & MIPI Port C on VLV/CHV */
+ i915_reg_t port_ctrl = IS_GEN9_LP(dev_priv) ?
+ BXT_MIPI_PORT_CTRL(port) : MIPI_PORT_CTRL(PORT_A);
+ u32 val;
+
+ I915_WRITE(MIPI_DEVICE_READY(port), DEVICE_READY |
+ ULPS_STATE_ENTER);
+ usleep_range(2000, 2500);
+
+ I915_WRITE(MIPI_DEVICE_READY(port), DEVICE_READY |
+ ULPS_STATE_EXIT);
+ usleep_range(2000, 2500);
+
+ I915_WRITE(MIPI_DEVICE_READY(port), DEVICE_READY |
+ ULPS_STATE_ENTER);
+ usleep_range(2000, 2500);
+
+ /*
+ * On VLV/CHV, wait till Clock lanes are in LP-00 state for MIPI
+ * Port A only. MIPI Port C has no similar bit for checking.
+ */
+ if ((IS_GEN9_LP(dev_priv) || port == PORT_A) &&
+ intel_wait_for_register(dev_priv,
+ port_ctrl, AFE_LATCHOUT, 0,
+ 30))
+ DRM_ERROR("DSI LP not going Low\n");
+
+ /* Disable MIPI PHY transparent latch */
+ val = I915_READ(port_ctrl);
+ I915_WRITE(port_ctrl, val & ~LP_OUTPUT_HOLD);
+ usleep_range(1000, 1500);
+
+ I915_WRITE(MIPI_DEVICE_READY(port), 0x00);
+ usleep_range(2000, 2500);
+ }
}
static void intel_dsi_port_enable(struct intel_encoder *encoder)
if (intel_dsi->dual_link == DSI_DUAL_LINK_FRONT_BACK) {
u32 temp;
-
- temp = I915_READ(VLV_CHICKEN_3);
- temp &= ~PIXEL_OVERLAP_CNT_MASK |
+ if (IS_GEN9_LP(dev_priv)) {
+ for_each_dsi_port(port, intel_dsi->ports) {
+ temp = I915_READ(MIPI_CTRL(port));
+ temp &= ~BXT_PIXEL_OVERLAP_CNT_MASK |
+ intel_dsi->pixel_overlap <<
+ BXT_PIXEL_OVERLAP_CNT_SHIFT;
+ I915_WRITE(MIPI_CTRL(port), temp);
+ }
+ } else {
+ temp = I915_READ(VLV_CHICKEN_3);
+ temp &= ~PIXEL_OVERLAP_CNT_MASK |
intel_dsi->pixel_overlap <<
PIXEL_OVERLAP_CNT_SHIFT;
- I915_WRITE(VLV_CHICKEN_3, temp);
+ I915_WRITE(VLV_CHICKEN_3, temp);
+ }
}
for_each_dsi_port(port, intel_dsi->ports) {
}
}
-static void intel_dsi_enable(struct intel_encoder *encoder)
-{
- struct drm_device *dev = encoder->base.dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
- enum port port;
-
- DRM_DEBUG_KMS("\n");
-
- if (is_cmd_mode(intel_dsi)) {
- for_each_dsi_port(port, intel_dsi->ports)
- I915_WRITE(MIPI_MAX_RETURN_PKT_SIZE(port), 8 * 4);
- } else {
- msleep(20); /* XXX */
- for_each_dsi_port(port, intel_dsi->ports)
- dpi_send_cmd(intel_dsi, TURN_ON, false, port);
- msleep(100);
-
- drm_panel_enable(intel_dsi->panel);
+static void intel_dsi_prepare(struct intel_encoder *intel_encoder,
+ struct intel_crtc_state *pipe_config);
+static void intel_dsi_unprepare(struct intel_encoder *encoder);
- for_each_dsi_port(port, intel_dsi->ports)
- wait_for_dsi_fifo_empty(intel_dsi, port);
+static void intel_dsi_msleep(struct intel_dsi *intel_dsi, int msec)
+{
+ struct drm_i915_private *dev_priv = to_i915(intel_dsi->base.base.dev);
- intel_dsi_port_enable(encoder);
- }
+ /* For v3 VBTs in vid-mode the delays are part of the VBT sequences */
+ if (is_vid_mode(intel_dsi) && dev_priv->vbt.dsi.seq_version >= 3)
+ return;
- intel_panel_enable_backlight(intel_dsi->attached_connector);
+ msleep(msec);
}
-static void intel_dsi_prepare(struct intel_encoder *intel_encoder,
- struct intel_crtc_state *pipe_config);
+/*
+ * Panel enable/disable sequences from the VBT spec.
+ *
+ * Note the spec has AssertReset / DeassertReset swapped from their
+ * usual naming. We use the normal names to avoid confusion (so below
+ * they are swapped compared to the spec).
+ *
+ * Steps starting with MIPI refer to VBT sequences, note that for v2
+ * VBTs several steps which have a VBT in v2 are expected to be handled
+ * directly by the driver, by directly driving gpios for example.
+ *
+ * v2 video mode seq v3 video mode seq command mode seq
+ * - power on - MIPIPanelPowerOn - power on
+ * - wait t1+t2 - wait t1+t2
+ * - MIPIDeassertResetPin - MIPIDeassertResetPin - MIPIDeassertResetPin
+ * - io lines to lp-11 - io lines to lp-11 - io lines to lp-11
+ * - MIPISendInitialDcsCmds - MIPISendInitialDcsCmds - MIPISendInitialDcsCmds
+ * - MIPITearOn
+ * - MIPIDisplayOn
+ * - turn on DPI - turn on DPI - set pipe to dsr mode
+ * - MIPIDisplayOn - MIPIDisplayOn
+ * - wait t5 - wait t5
+ * - backlight on - MIPIBacklightOn - backlight on
+ * ... ... ... issue mem cmds ...
+ * - backlight off - MIPIBacklightOff - backlight off
+ * - wait t6 - wait t6
+ * - MIPIDisplayOff
+ * - turn off DPI - turn off DPI - disable pipe dsr mode
+ * - MIPITearOff
+ * - MIPIDisplayOff - MIPIDisplayOff
+ * - io lines to lp-00 - io lines to lp-00 - io lines to lp-00
+ * - MIPIAssertResetPin - MIPIAssertResetPin - MIPIAssertResetPin
+ * - wait t3 - wait t3
+ * - power off - MIPIPanelPowerOff - power off
+ * - wait t4 - wait t4
+ */
static void intel_dsi_pre_enable(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config,
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
enum port port;
+ u32 val;
DRM_DEBUG_KMS("\n");
intel_disable_dsi_pll(encoder);
intel_enable_dsi_pll(encoder, pipe_config);
- intel_dsi_prepare(encoder, pipe_config);
-
- /* Panel Enable over CRC PMIC */
- if (intel_dsi->gpio_panel)
- gpiod_set_value_cansleep(intel_dsi->gpio_panel, 1);
+ if (IS_BROXTON(dev_priv)) {
+ /* Add MIPI IO reset programming for modeset */
+ val = I915_READ(BXT_P_CR_GT_DISP_PWRON);
+ I915_WRITE(BXT_P_CR_GT_DISP_PWRON,
+ val | MIPIO_RST_CTRL);
- msleep(intel_dsi->panel_on_delay);
+ /* Power up DSI regulator */
+ I915_WRITE(BXT_P_DSI_REGULATOR_CFG, STAP_SELECT);
+ I915_WRITE(BXT_P_DSI_REGULATOR_TX_CTRL, 0);
+ }
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
u32 val;
I915_WRITE(DSPCLK_GATE_D, val);
}
- /* put device in ready state */
- intel_dsi_device_ready(encoder);
+ intel_dsi_prepare(encoder, pipe_config);
- drm_panel_prepare(intel_dsi->panel);
+ /* Power on, try both CRC pmic gpio and VBT */
+ if (intel_dsi->gpio_panel)
+ gpiod_set_value_cansleep(intel_dsi->gpio_panel, 1);
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_POWER_ON);
+ intel_dsi_msleep(intel_dsi, intel_dsi->panel_on_delay);
+
+ /* Deassert reset */
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_DEASSERT_RESET);
- for_each_dsi_port(port, intel_dsi->ports)
- wait_for_dsi_fifo_empty(intel_dsi, port);
+ /* Put device in ready state (LP-11) */
+ intel_dsi_device_ready(encoder);
+
+ /* Send initialization commands in LP mode */
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_INIT_OTP);
/* Enable port in pre-enable phase itself because as per hw team
* recommendation, port should be enabled befor plane & pipe */
- intel_dsi_enable(encoder);
+ if (is_cmd_mode(intel_dsi)) {
+ for_each_dsi_port(port, intel_dsi->ports)
+ I915_WRITE(MIPI_MAX_RETURN_PKT_SIZE(port), 8 * 4);
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_TEAR_ON);
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_DISPLAY_ON);
+ } else {
+ msleep(20); /* XXX */
+ for_each_dsi_port(port, intel_dsi->ports)
+ dpi_send_cmd(intel_dsi, TURN_ON, false, port);
+ intel_dsi_msleep(intel_dsi, 100);
+
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_DISPLAY_ON);
+
+ intel_dsi_port_enable(encoder);
+ }
+
+ intel_panel_enable_backlight(intel_dsi->attached_connector);
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_BACKLIGHT_ON);
}
+/*
+ * DSI port enable has to be done before pipe and plane enable, so we do it in
+ * the pre_enable hook.
+ */
static void intel_dsi_enable_nop(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config,
struct drm_connector_state *conn_state)
{
DRM_DEBUG_KMS("\n");
-
- /* for DSI port enable has to be done before pipe
- * and plane enable, so port enable is done in
- * pre_enable phase itself unlike other encoders
- */
}
-static void intel_dsi_pre_disable(struct intel_encoder *encoder,
- struct intel_crtc_state *old_crtc_state,
- struct drm_connector_state *old_conn_state)
+/*
+ * DSI port disable has to be done after pipe and plane disable, so we do it in
+ * the post_disable hook.
+ */
+static void intel_dsi_disable(struct intel_encoder *encoder,
+ struct intel_crtc_state *old_crtc_state,
+ struct drm_connector_state *old_conn_state)
{
+ struct drm_device *dev = encoder->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
enum port port;
DRM_DEBUG_KMS("\n");
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_BACKLIGHT_OFF);
intel_panel_disable_backlight(intel_dsi->attached_connector);
+ /*
+ * Disable Device ready before the port shutdown in order
+ * to avoid split screen
+ */
+ if (IS_BROXTON(dev_priv)) {
+ for_each_dsi_port(port, intel_dsi->ports)
+ I915_WRITE(MIPI_DEVICE_READY(port), 0);
+ }
+
+ /*
+ * According to the spec we should send SHUTDOWN before
+ * MIPI_SEQ_DISPLAY_OFF only for v3+ VBTs, but field testing
+ * has shown that the v3 sequence works for v2 VBTs too
+ */
if (is_vid_mode(intel_dsi)) {
/* Send Shutdown command to the panel in LP mode */
for_each_dsi_port(port, intel_dsi->ports)
}
}
-static void intel_dsi_disable(struct intel_encoder *encoder)
+static void intel_dsi_clear_device_ready(struct intel_encoder *encoder)
{
- struct drm_device *dev = encoder->base.dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+
+ if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv) ||
+ IS_BROXTON(dev_priv))
+ vlv_dsi_clear_device_ready(encoder);
+ else if (IS_GEMINILAKE(dev_priv))
+ glk_dsi_clear_device_ready(encoder);
+}
+
+static void intel_dsi_post_disable(struct intel_encoder *encoder,
+ struct intel_crtc_state *pipe_config,
+ struct drm_connector_state *conn_state)
+{
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
enum port port;
- u32 temp;
+ u32 val;
DRM_DEBUG_KMS("\n");
wait_for_dsi_fifo_empty(intel_dsi, port);
intel_dsi_port_disable(encoder);
- msleep(2);
+ usleep_range(2000, 5000);
}
- for_each_dsi_port(port, intel_dsi->ports) {
- /* Panel commands can be sent when clock is in LP11 */
- I915_WRITE(MIPI_DEVICE_READY(port), 0x0);
-
- intel_dsi_reset_clocks(encoder, port);
- I915_WRITE(MIPI_EOT_DISABLE(port), CLOCKSTOP);
+ intel_dsi_unprepare(encoder);
- temp = I915_READ(MIPI_DSI_FUNC_PRG(port));
- temp &= ~VID_MODE_FORMAT_MASK;
- I915_WRITE(MIPI_DSI_FUNC_PRG(port), temp);
-
- I915_WRITE(MIPI_DEVICE_READY(port), 0x1);
- }
- /* if disable packets are sent before sending shutdown packet then in
- * some next enable sequence send turn on packet error is observed */
- drm_panel_disable(intel_dsi->panel);
-
- for_each_dsi_port(port, intel_dsi->ports)
- wait_for_dsi_fifo_empty(intel_dsi, port);
-}
-
-static void intel_dsi_clear_device_ready(struct intel_encoder *encoder)
-{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
- enum port port;
-
- DRM_DEBUG_KMS("\n");
- for_each_dsi_port(port, intel_dsi->ports) {
- /* Common bit for both MIPI Port A & MIPI Port C on VLV/CHV */
- i915_reg_t port_ctrl = IS_GEN9_LP(dev_priv) ?
- BXT_MIPI_PORT_CTRL(port) : MIPI_PORT_CTRL(PORT_A);
- u32 val;
-
- I915_WRITE(MIPI_DEVICE_READY(port), DEVICE_READY |
- ULPS_STATE_ENTER);
- usleep_range(2000, 2500);
-
- I915_WRITE(MIPI_DEVICE_READY(port), DEVICE_READY |
- ULPS_STATE_EXIT);
- usleep_range(2000, 2500);
-
- I915_WRITE(MIPI_DEVICE_READY(port), DEVICE_READY |
- ULPS_STATE_ENTER);
- usleep_range(2000, 2500);
+ /*
+ * if disable packets are sent before sending shutdown packet then in
+ * some next enable sequence send turn on packet error is observed
+ */
+ if (is_cmd_mode(intel_dsi))
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_TEAR_OFF);
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_DISPLAY_OFF);
- /* Wait till Clock lanes are in LP-00 state for MIPI Port A
- * only. MIPI Port C has no similar bit for checking
- */
- if (intel_wait_for_register(dev_priv,
- port_ctrl, AFE_LATCHOUT, 0,
- 30))
- DRM_ERROR("DSI LP not going Low\n");
+ /* Transition to LP-00 */
+ intel_dsi_clear_device_ready(encoder);
- /* Disable MIPI PHY transparent latch */
- val = I915_READ(port_ctrl);
- I915_WRITE(port_ctrl, val & ~LP_OUTPUT_HOLD);
- usleep_range(1000, 1500);
+ if (IS_BROXTON(dev_priv)) {
+ /* Power down DSI regulator to save power */
+ I915_WRITE(BXT_P_DSI_REGULATOR_CFG, STAP_SELECT);
+ I915_WRITE(BXT_P_DSI_REGULATOR_TX_CTRL, HS_IO_CTRL_SELECT);
- I915_WRITE(MIPI_DEVICE_READY(port), 0x00);
- usleep_range(2000, 2500);
+ /* Add MIPI IO reset programming for modeset */
+ val = I915_READ(BXT_P_CR_GT_DISP_PWRON);
+ I915_WRITE(BXT_P_CR_GT_DISP_PWRON,
+ val & ~MIPIO_RST_CTRL);
}
-}
-
-static void intel_dsi_post_disable(struct intel_encoder *encoder,
- struct intel_crtc_state *pipe_config,
- struct drm_connector_state *conn_state)
-{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
-
- DRM_DEBUG_KMS("\n");
-
- intel_dsi_disable(encoder);
-
- intel_dsi_clear_device_ready(encoder);
intel_disable_dsi_pll(encoder);
I915_WRITE(DSPCLK_GATE_D, val);
}
- drm_panel_unprepare(intel_dsi->panel);
+ /* Assert reset */
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_ASSERT_RESET);
- msleep(intel_dsi->panel_off_delay);
-
- /* Panel Disable over CRC PMIC */
+ /* Power off, try both CRC pmic gpio and VBT */
+ intel_dsi_msleep(intel_dsi, intel_dsi->panel_off_delay);
+ intel_dsi_vbt_exec_sequence(intel_dsi, MIPI_SEQ_POWER_OFF);
if (intel_dsi->gpio_panel)
gpiod_set_value_cansleep(intel_dsi->gpio_panel, 0);
* FIXME As we do with eDP, just make a note of the time here
* and perform the wait before the next panel power on.
*/
- msleep(intel_dsi->panel_pwr_cycle_delay);
+ intel_dsi_msleep(intel_dsi, intel_dsi->panel_pwr_cycle_delay);
}
static bool intel_dsi_get_hw_state(struct intel_encoder *encoder,
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
- enum intel_display_power_domain power_domain;
enum port port;
bool active = false;
DRM_DEBUG_KMS("\n");
- power_domain = intel_display_port_power_domain(encoder);
- if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
+ if (!intel_display_power_get_if_enabled(dev_priv,
+ encoder->power_domain))
return false;
/*
}
out_put_power:
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, encoder->power_domain);
return active;
}
*/
I915_WRITE(MIPI_LP_BYTECLK(port), intel_dsi->lp_byte_clk);
+ if (IS_GEMINILAKE(dev_priv)) {
+ I915_WRITE(MIPI_TLPX_TIME_COUNT(port),
+ intel_dsi->lp_byte_clk);
+ /* Shadow of DPHY reg */
+ I915_WRITE(MIPI_CLK_LANE_TIMING(port),
+ intel_dsi->dphy_reg);
+ }
+
/* the bw essential for transmitting 16 long packets containing
* 252 bytes meant for dcs write memory command is programmed in
* this register in terms of byte clocks. based on dsi transfer
}
}
+static void intel_dsi_unprepare(struct intel_encoder *encoder)
+{
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
+ enum port port;
+ u32 val;
+
+ if (!IS_GEMINILAKE(dev_priv)) {
+ for_each_dsi_port(port, intel_dsi->ports) {
+ /* Panel commands can be sent when clock is in LP11 */
+ I915_WRITE(MIPI_DEVICE_READY(port), 0x0);
+
+ intel_dsi_reset_clocks(encoder, port);
+ I915_WRITE(MIPI_EOT_DISABLE(port), CLOCKSTOP);
+
+ val = I915_READ(MIPI_DSI_FUNC_PRG(port));
+ val &= ~VID_MODE_FORMAT_MASK;
+ I915_WRITE(MIPI_DSI_FUNC_PRG(port), val);
+
+ I915_WRITE(MIPI_DEVICE_READY(port), 0x1);
+ }
+ }
+}
+
static int intel_dsi_get_modes(struct drm_connector *connector)
{
struct intel_connector *intel_connector = to_intel_connector(connector);
{
struct intel_dsi *intel_dsi = enc_to_intel_dsi(encoder);
- if (intel_dsi->panel) {
- drm_panel_detach(intel_dsi->panel);
- /* XXX: Logically this call belongs in the panel driver. */
- drm_panel_remove(intel_dsi->panel);
- }
-
/* dispose of the gpios */
if (intel_dsi->gpio_panel)
gpiod_put(intel_dsi->gpio_panel);
struct drm_connector *connector;
struct drm_display_mode *scan, *fixed_mode = NULL;
enum port port;
- unsigned int i;
DRM_DEBUG_KMS("\n");
intel_encoder->compute_config = intel_dsi_compute_config;
intel_encoder->pre_enable = intel_dsi_pre_enable;
intel_encoder->enable = intel_dsi_enable_nop;
- intel_encoder->disable = intel_dsi_pre_disable;
+ intel_encoder->disable = intel_dsi_disable;
intel_encoder->post_disable = intel_dsi_post_disable;
intel_encoder->get_hw_state = intel_dsi_get_hw_state;
intel_encoder->get_config = intel_dsi_get_config;
intel_connector->get_hw_state = intel_connector_get_hw_state;
intel_encoder->port = port;
+
/*
* On BYT/CHV, pipe A maps to MIPI DSI port A, pipe B maps to MIPI DSI
* port C. BXT isn't limited like this.
intel_dsi->dsi_hosts[port] = host;
}
- for (i = 0; i < ARRAY_SIZE(intel_dsi_drivers); i++) {
- intel_dsi->panel = intel_dsi_drivers[i].init(intel_dsi,
- intel_dsi_drivers[i].panel_id);
- if (intel_dsi->panel)
- break;
- }
-
- if (!intel_dsi->panel) {
+ if (!intel_dsi_vbt_init(intel_dsi, MIPI_DSI_GENERIC_PANEL_ID)) {
DRM_DEBUG_KMS("no device found\n");
goto err;
}
* In case of BYT with CRC PMIC, we need to use GPIO for
* Panel control.
*/
- if (dev_priv->vbt.dsi.config->pwm_blc == PPS_BLC_PMIC) {
+ if ((IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) &&
+ (dev_priv->vbt.dsi.config->pwm_blc == PPS_BLC_PMIC)) {
intel_dsi->gpio_panel =
gpiod_get(dev->dev, "panel", GPIOD_OUT_HIGH);
}
intel_encoder->type = INTEL_OUTPUT_DSI;
+ intel_encoder->power_domain = POWER_DOMAIN_PORT_DSI;
intel_encoder->cloneable = 0;
drm_connector_init(dev, connector, &intel_dsi_connector_funcs,
DRM_MODE_CONNECTOR_DSI);
intel_connector_attach_encoder(intel_connector, intel_encoder);
- drm_panel_attach(intel_dsi->panel, connector);
-
mutex_lock(&dev->mode_config.mutex);
- drm_panel_get_modes(intel_dsi->panel);
+ intel_dsi_vbt_get_modes(intel_dsi);
list_for_each_entry(scan, &connector->probed_modes, head) {
if ((scan->type & DRM_MODE_TYPE_PREFERRED)) {
fixed_mode = drm_mode_duplicate(dev, scan);
struct intel_dsi {
struct intel_encoder base;
- struct drm_panel *panel;
struct intel_dsi_host *dsi_hosts[I915_MAX_PORTS];
/* GPIO Desc for CRC based Panel control */
return container_of(encoder, struct intel_dsi, base.base);
}
+/* intel_dsi.c */
+void wait_for_dsi_fifo_empty(struct intel_dsi *intel_dsi, enum port port);
+enum mipi_dsi_pixel_format pixel_format_from_register_bits(u32 fmt);
+
+/* intel_dsi_pll.c */
bool intel_dsi_pll_is_enabled(struct drm_i915_private *dev_priv);
int intel_compute_dsi_pll(struct intel_encoder *encoder,
struct intel_crtc_state *config);
void intel_dsi_reset_clocks(struct intel_encoder *encoder,
enum port port);
-struct drm_panel *vbt_panel_init(struct intel_dsi *intel_dsi, u16 panel_id);
-enum mipi_dsi_pixel_format pixel_format_from_register_bits(u32 fmt);
+/* intel_dsi_vbt.c */
+bool intel_dsi_vbt_init(struct intel_dsi *intel_dsi, u16 panel_id);
+int intel_dsi_vbt_get_modes(struct intel_dsi *intel_dsi);
+void intel_dsi_vbt_exec_sequence(struct intel_dsi *intel_dsi,
+ enum mipi_seq seq_id);
#endif /* _INTEL_DSI_H */
return false;
/*
- * Both dividers must be programmed with valid values even if only one
- * of the PLL is used, see BSpec/Broxton Clocks. Check this here for
+ * Dividers must be programmed with valid values. As per BSEPC, for
+ * GEMINLAKE only PORT A divider values are checked while for BXT
+ * both divider values are validated. Check this here for
* paranoia, since BIOS is known to misconfigure PLLs in this way at
* times, and since accessing DSI registers with invalid dividers
* causes a system hang.
*/
val = I915_READ(BXT_DSI_PLL_CTL);
- if (!(val & BXT_DSIA_16X_MASK) || !(val & BXT_DSIC_16X_MASK)) {
- DRM_DEBUG_DRIVER("PLL is enabled with invalid divider settings (%08x)\n",
- val);
- enabled = false;
+ if (IS_GEMINILAKE(dev_priv)) {
+ if (!(val & BXT_DSIA_16X_MASK)) {
+ DRM_DEBUG_DRIVER("Invalid PLL divider (%08x)\n", val);
+ enabled = false;
+ }
+ } else {
+ if (!(val & BXT_DSIA_16X_MASK) || !(val & BXT_DSIC_16X_MASK)) {
+ DRM_DEBUG_DRIVER("Invalid PLL divider (%08x)\n", val);
+ enabled = false;
+ }
}
return enabled;
ESCAPE_CLOCK_DIVIDER_SHIFT);
}
+static void glk_dsi_program_esc_clock(struct drm_device *dev,
+ const struct intel_crtc_state *config)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ u32 dsi_rate = 0;
+ u32 pll_ratio = 0;
+ u32 ddr_clk = 0;
+ u32 div1_value = 0;
+ u32 div2_value = 0;
+ u32 txesc1_div = 0;
+ u32 txesc2_div = 0;
+
+ pll_ratio = config->dsi_pll.ctrl & BXT_DSI_PLL_RATIO_MASK;
+
+ dsi_rate = (BXT_REF_CLOCK_KHZ * pll_ratio) / 2;
+
+ ddr_clk = dsi_rate / 2;
+
+ /* Variable divider value */
+ div1_value = DIV_ROUND_CLOSEST(ddr_clk, 20000);
+
+ /* Calculate TXESC1 divider */
+ if (div1_value <= 10)
+ txesc1_div = div1_value;
+ else if ((div1_value > 10) && (div1_value <= 20))
+ txesc1_div = DIV_ROUND_UP(div1_value, 2);
+ else if ((div1_value > 20) && (div1_value <= 30))
+ txesc1_div = DIV_ROUND_UP(div1_value, 4);
+ else if ((div1_value > 30) && (div1_value <= 40))
+ txesc1_div = DIV_ROUND_UP(div1_value, 6);
+ else if ((div1_value > 40) && (div1_value <= 50))
+ txesc1_div = DIV_ROUND_UP(div1_value, 8);
+ else
+ txesc1_div = 10;
+
+ /* Calculate TXESC2 divider */
+ div2_value = DIV_ROUND_UP(div1_value, txesc1_div);
+
+ if (div2_value < 10)
+ txesc2_div = div2_value;
+ else
+ txesc2_div = 10;
+
+ I915_WRITE(MIPIO_TXESC_CLK_DIV1, txesc1_div & GLK_TX_ESC_CLK_DIV1_MASK);
+ I915_WRITE(MIPIO_TXESC_CLK_DIV2, txesc2_div & GLK_TX_ESC_CLK_DIV2_MASK);
+}
+
/* Program BXT Mipi clocks and dividers */
static void bxt_dsi_program_clocks(struct drm_device *dev, enum port port,
const struct intel_crtc_state *config)
rx_div_lower = rx_div & RX_DIVIDER_BIT_1_2;
rx_div_upper = (rx_div & RX_DIVIDER_BIT_3_4) >> 2;
- /* As per bpsec program the 8/3X clock divider to the below value */
- if (dev_priv->vbt.dsi.config->is_cmd_mode)
- mipi_8by3_divider = 0x2;
- else
- mipi_8by3_divider = 0x3;
+ mipi_8by3_divider = 0x2;
tmp |= BXT_MIPI_8X_BY3_DIVIDER(port, mipi_8by3_divider);
tmp |= BXT_MIPI_TX_ESCLK_DIVIDER(port, tx_div);
I915_WRITE(BXT_MIPI_CLOCK_CTL, tmp);
}
-static int bxt_compute_dsi_pll(struct intel_encoder *encoder,
+static int gen9lp_compute_dsi_pll(struct intel_encoder *encoder,
struct intel_crtc_state *config)
{
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
- u8 dsi_ratio;
+ u8 dsi_ratio, dsi_ratio_min, dsi_ratio_max;
u32 dsi_clk;
dsi_clk = dsi_clk_from_pclk(intel_dsi->pclk, intel_dsi->pixel_format,
* round 'up' the result
*/
dsi_ratio = DIV_ROUND_UP(dsi_clk * 2, BXT_REF_CLOCK_KHZ);
- if (dsi_ratio < BXT_DSI_PLL_RATIO_MIN ||
- dsi_ratio > BXT_DSI_PLL_RATIO_MAX) {
+
+ if (IS_BROXTON(dev_priv)) {
+ dsi_ratio_min = BXT_DSI_PLL_RATIO_MIN;
+ dsi_ratio_max = BXT_DSI_PLL_RATIO_MAX;
+ } else {
+ dsi_ratio_min = GLK_DSI_PLL_RATIO_MIN;
+ dsi_ratio_max = GLK_DSI_PLL_RATIO_MAX;
+ }
+
+ if (dsi_ratio < dsi_ratio_min || dsi_ratio > dsi_ratio_max) {
DRM_ERROR("Cant get a suitable ratio from DSI PLL ratios\n");
return -ECHRNG;
- }
+ } else
+ DRM_DEBUG_KMS("DSI PLL calculation is Done!!\n");
/*
* Program DSI ratio and Select MIPIC and MIPIA PLL output as 8x
/* As per recommendation from hardware team,
* Prog PVD ratio =1 if dsi ratio <= 50
*/
- if (dsi_ratio <= 50)
+ if (IS_BROXTON(dev_priv) && dsi_ratio <= 50)
config->dsi_pll.ctrl |= BXT_DSI_PLL_PVD_RATIO_1;
return 0;
}
-static void bxt_enable_dsi_pll(struct intel_encoder *encoder,
+static void gen9lp_enable_dsi_pll(struct intel_encoder *encoder,
const struct intel_crtc_state *config)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
POSTING_READ(BXT_DSI_PLL_CTL);
/* Program TX, RX, Dphy clocks */
- for_each_dsi_port(port, intel_dsi->ports)
- bxt_dsi_program_clocks(encoder->base.dev, port, config);
+ if (IS_BROXTON(dev_priv)) {
+ for_each_dsi_port(port, intel_dsi->ports)
+ bxt_dsi_program_clocks(encoder->base.dev, port, config);
+ } else {
+ glk_dsi_program_esc_clock(encoder->base.dev, config);
+ }
/* Enable DSI PLL */
val = I915_READ(BXT_DSI_PLL_ENABLE);
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
return vlv_compute_dsi_pll(encoder, config);
else if (IS_GEN9_LP(dev_priv))
- return bxt_compute_dsi_pll(encoder, config);
+ return gen9lp_compute_dsi_pll(encoder, config);
return -ENODEV;
}
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
vlv_enable_dsi_pll(encoder, config);
else if (IS_GEN9_LP(dev_priv))
- bxt_enable_dsi_pll(encoder, config);
+ gen9lp_enable_dsi_pll(encoder, config);
}
void intel_disable_dsi_pll(struct intel_encoder *encoder)
bxt_disable_dsi_pll(encoder);
}
-static void bxt_dsi_reset_clocks(struct intel_encoder *encoder, enum port port)
+static void gen9lp_dsi_reset_clocks(struct intel_encoder *encoder,
+ enum port port)
{
u32 tmp;
struct drm_device *dev = encoder->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
/* Clear old configurations */
- tmp = I915_READ(BXT_MIPI_CLOCK_CTL);
- tmp &= ~(BXT_MIPI_TX_ESCLK_FIXDIV_MASK(port));
- tmp &= ~(BXT_MIPI_RX_ESCLK_UPPER_FIXDIV_MASK(port));
- tmp &= ~(BXT_MIPI_8X_BY3_DIVIDER_MASK(port));
- tmp &= ~(BXT_MIPI_RX_ESCLK_LOWER_FIXDIV_MASK(port));
- I915_WRITE(BXT_MIPI_CLOCK_CTL, tmp);
+ if (IS_BROXTON(dev_priv)) {
+ tmp = I915_READ(BXT_MIPI_CLOCK_CTL);
+ tmp &= ~(BXT_MIPI_TX_ESCLK_FIXDIV_MASK(port));
+ tmp &= ~(BXT_MIPI_RX_ESCLK_UPPER_FIXDIV_MASK(port));
+ tmp &= ~(BXT_MIPI_8X_BY3_DIVIDER_MASK(port));
+ tmp &= ~(BXT_MIPI_RX_ESCLK_LOWER_FIXDIV_MASK(port));
+ I915_WRITE(BXT_MIPI_CLOCK_CTL, tmp);
+ } else {
+ tmp = I915_READ(MIPIO_TXESC_CLK_DIV1);
+ tmp &= ~GLK_TX_ESC_CLK_DIV1_MASK;
+ I915_WRITE(MIPIO_TXESC_CLK_DIV1, tmp);
+
+ tmp = I915_READ(MIPIO_TXESC_CLK_DIV2);
+ tmp &= ~GLK_TX_ESC_CLK_DIV2_MASK;
+ I915_WRITE(MIPIO_TXESC_CLK_DIV2, tmp);
+ }
I915_WRITE(MIPI_EOT_DISABLE(port), CLOCKSTOP);
}
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
if (IS_GEN9_LP(dev_priv))
- bxt_dsi_reset_clocks(encoder, port);
+ gen9lp_dsi_reset_clocks(encoder, port);
else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
vlv_dsi_reset_clocks(encoder, port);
}
#include <drm/drm_crtc.h>
#include <drm/drm_edid.h>
#include <drm/i915_drm.h>
-#include <drm/drm_panel.h>
#include <linux/gpio/consumer.h>
#include <linux/slab.h>
#include <video/mipi_display.h>
#include "intel_drv.h"
#include "intel_dsi.h"
-struct vbt_panel {
- struct drm_panel panel;
- struct intel_dsi *intel_dsi;
-};
-
-static inline struct vbt_panel *to_vbt_panel(struct drm_panel *panel)
-{
- return container_of(panel, struct vbt_panel, panel);
-}
-
#define MIPI_TRANSFER_MODE_SHIFT 0
#define MIPI_VIRTUAL_CHANNEL_SHIFT 1
#define MIPI_PORT_SHIFT 3
break;
}
+ wait_for_dsi_fifo_empty(intel_dsi, port);
+
out:
data += len;
return "(unknown)";
}
-static void generic_exec_sequence(struct drm_panel *panel, enum mipi_seq seq_id)
+void intel_dsi_vbt_exec_sequence(struct intel_dsi *intel_dsi,
+ enum mipi_seq seq_id)
{
- struct vbt_panel *vbt_panel = to_vbt_panel(panel);
- struct intel_dsi *intel_dsi = vbt_panel->intel_dsi;
struct drm_i915_private *dev_priv = to_i915(intel_dsi->base.base.dev);
const u8 *data;
fn_mipi_elem_exec mipi_elem_exec;
}
}
-static int vbt_panel_prepare(struct drm_panel *panel)
+int intel_dsi_vbt_get_modes(struct intel_dsi *intel_dsi)
{
- generic_exec_sequence(panel, MIPI_SEQ_ASSERT_RESET);
- generic_exec_sequence(panel, MIPI_SEQ_POWER_ON);
- generic_exec_sequence(panel, MIPI_SEQ_DEASSERT_RESET);
- generic_exec_sequence(panel, MIPI_SEQ_INIT_OTP);
-
- return 0;
-}
-
-static int vbt_panel_unprepare(struct drm_panel *panel)
-{
- generic_exec_sequence(panel, MIPI_SEQ_ASSERT_RESET);
- generic_exec_sequence(panel, MIPI_SEQ_POWER_OFF);
-
- return 0;
-}
-
-static int vbt_panel_enable(struct drm_panel *panel)
-{
- generic_exec_sequence(panel, MIPI_SEQ_DISPLAY_ON);
- generic_exec_sequence(panel, MIPI_SEQ_BACKLIGHT_ON);
-
- return 0;
-}
-
-static int vbt_panel_disable(struct drm_panel *panel)
-{
- generic_exec_sequence(panel, MIPI_SEQ_BACKLIGHT_OFF);
- generic_exec_sequence(panel, MIPI_SEQ_DISPLAY_OFF);
-
- return 0;
-}
-
-static int vbt_panel_get_modes(struct drm_panel *panel)
-{
- struct vbt_panel *vbt_panel = to_vbt_panel(panel);
- struct intel_dsi *intel_dsi = vbt_panel->intel_dsi;
+ struct intel_connector *connector = intel_dsi->attached_connector;
struct drm_device *dev = intel_dsi->base.base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_display_mode *mode;
- if (!panel->connector)
- return 0;
-
mode = drm_mode_duplicate(dev, dev_priv->vbt.lfp_lvds_vbt_mode);
if (!mode)
return 0;
mode->type |= DRM_MODE_TYPE_PREFERRED;
- drm_mode_probed_add(panel->connector, mode);
+ drm_mode_probed_add(&connector->base, mode);
return 1;
}
-static const struct drm_panel_funcs vbt_panel_funcs = {
- .disable = vbt_panel_disable,
- .unprepare = vbt_panel_unprepare,
- .prepare = vbt_panel_prepare,
- .enable = vbt_panel_enable,
- .get_modes = vbt_panel_get_modes,
-};
-
-struct drm_panel *vbt_panel_init(struct intel_dsi *intel_dsi, u16 panel_id)
+bool intel_dsi_vbt_init(struct intel_dsi *intel_dsi, u16 panel_id)
{
struct drm_device *dev = intel_dsi->base.base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct mipi_config *mipi_config = dev_priv->vbt.dsi.config;
struct mipi_pps_data *pps = dev_priv->vbt.dsi.pps;
struct drm_display_mode *mode = dev_priv->vbt.lfp_lvds_vbt_mode;
- struct vbt_panel *vbt_panel;
u32 bpp;
u32 tlpx_ns, extra_byte_count, bitrate, tlpx_ui;
u32 ui_num, ui_den;
u32 tclk_prepare_clkzero, ths_prepare_hszero;
u32 lp_to_hs_switch, hs_to_lp_switch;
u32 pclk, computed_ddr;
+ u32 mul;
u16 burst_mode_ratio;
enum port port;
if (mipi_config->target_burst_mode_freq <
computed_ddr) {
DRM_ERROR("Burst mode freq is less than computed\n");
- return NULL;
+ return false;
}
burst_mode_ratio = DIV_ROUND_UP(
pclk = DIV_ROUND_UP(pclk * burst_mode_ratio, 100);
} else {
DRM_ERROR("Burst mode target is not set\n");
- return NULL;
+ return false;
}
} else
burst_mode_ratio = 100;
break;
}
- /*
- * ui(s) = 1/f [f in hz]
- * ui(ns) = 10^9 / (f*10^6) [f in Mhz] -> 10^3/f(Mhz)
- */
-
/* in Kbps */
ui_num = NS_KHZ_RATIO;
ui_den = bitrate;
*/
intel_dsi->lp_byte_clk = DIV_ROUND_UP(tlpx_ns * ui_den, 8 * ui_num);
- /* count values in UI = (ns value) * (bitrate / (2 * 10^6))
+ /* DDR clock period = 2 * UI
+ * UI(sec) = 1/(bitrate * 10^3) (bitrate is in KHZ)
+ * UI(nsec) = 10^6 / bitrate
+ * DDR clock period (nsec) = 2 * UI = (2 * 10^6)/ bitrate
+ * DDR clock count = ns_value / DDR clock period
*
- * Since txddrclkhs_i is 2xUI, all the count values programmed in
- * DPHY param register are divided by 2
- *
- * prepare count
+ * For GEMINILAKE dphy_param_reg will be programmed in terms of
+ * HS byte clock count for other platform in HS ddr clock count
*/
+ mul = IS_GEMINILAKE(dev_priv) ? 8 : 2;
ths_prepare_ns = max(mipi_config->ths_prepare,
mipi_config->tclk_prepare);
- prepare_cnt = DIV_ROUND_UP(ths_prepare_ns * ui_den, ui_num * 2);
+
+ /* prepare count */
+ prepare_cnt = DIV_ROUND_UP(ths_prepare_ns * ui_den, ui_num * mul);
/* exit zero count */
exit_zero_cnt = DIV_ROUND_UP(
(ths_prepare_hszero - ths_prepare_ns) * ui_den,
- ui_num * 2
+ ui_num * mul
);
/*
/* clk zero count */
clk_zero_cnt = DIV_ROUND_UP(
- (tclk_prepare_clkzero - ths_prepare_ns)
- * ui_den, 2 * ui_num);
+ (tclk_prepare_clkzero - ths_prepare_ns)
+ * ui_den, ui_num * mul);
/* trail count */
tclk_trail_ns = max(mipi_config->tclk_trail, mipi_config->ths_trail);
- trail_cnt = DIV_ROUND_UP(tclk_trail_ns * ui_den, 2 * ui_num);
+ trail_cnt = DIV_ROUND_UP(tclk_trail_ns * ui_den, ui_num * mul);
if (prepare_cnt > PREPARE_CNT_MAX ||
exit_zero_cnt > EXIT_ZERO_CNT_MAX ||
8);
intel_dsi->clk_hs_to_lp_count += extra_byte_count;
+ DRM_DEBUG_KMS("Pclk %d\n", intel_dsi->pclk);
+ DRM_DEBUG_KMS("Pixel overlap %d\n", intel_dsi->pixel_overlap);
+ DRM_DEBUG_KMS("Lane count %d\n", intel_dsi->lane_count);
+ DRM_DEBUG_KMS("DPHY param reg 0x%x\n", intel_dsi->dphy_reg);
+ DRM_DEBUG_KMS("Video mode format %s\n",
+ intel_dsi->video_mode_format == VIDEO_MODE_NON_BURST_WITH_SYNC_PULSE ?
+ "non-burst with sync pulse" :
+ intel_dsi->video_mode_format == VIDEO_MODE_NON_BURST_WITH_SYNC_EVENTS ?
+ "non-burst with sync events" :
+ intel_dsi->video_mode_format == VIDEO_MODE_BURST ?
+ "burst" : "<unknown>");
+ DRM_DEBUG_KMS("Burst mode ratio %d\n", intel_dsi->burst_mode_ratio);
+ DRM_DEBUG_KMS("Reset timer %d\n", intel_dsi->rst_timer_val);
DRM_DEBUG_KMS("Eot %s\n", enableddisabled(intel_dsi->eotp_pkt));
DRM_DEBUG_KMS("Clockstop %s\n", enableddisabled(!intel_dsi->clock_stop));
DRM_DEBUG_KMS("Mode %s\n", intel_dsi->operation_mode ? "command" : "video");
intel_dsi->panel_off_delay = pps->panel_off_delay / 10;
intel_dsi->panel_pwr_cycle_delay = pps->panel_power_cycle_delay / 10;
- /* This is cheating a bit with the cleanup. */
- vbt_panel = devm_kzalloc(dev->dev, sizeof(*vbt_panel), GFP_KERNEL);
- if (!vbt_panel)
- return NULL;
-
- vbt_panel->intel_dsi = intel_dsi;
- drm_panel_init(&vbt_panel->panel);
- vbt_panel->panel.funcs = &vbt_panel_funcs;
- drm_panel_add(&vbt_panel->panel);
-
/* a regular driver would get the device in probe */
for_each_dsi_port(port, intel_dsi->ports) {
mipi_dsi_attach(intel_dsi->dsi_hosts[port]->device);
}
- return &vbt_panel->panel;
+ return true;
}
"DVO %c", port_name(port));
intel_encoder->type = INTEL_OUTPUT_DVO;
+ intel_encoder->power_domain = POWER_DOMAIN_PORT_OTHER;
intel_encoder->port = port;
intel_encoder->crtc_mask = (1 << 0) | (1 << 1);
static const struct engine_info {
const char *name;
- unsigned exec_id;
- enum intel_engine_hw_id hw_id;
+ unsigned int exec_id;
+ unsigned int hw_id;
u32 mmio_base;
unsigned irq_shift;
int (*init_legacy)(struct intel_engine_cs *engine);
}
/**
- * intel_engines_init() - allocate, populate and init the Engine Command Streamers
+ * intel_engines_init_early() - allocate the Engine Command Streamers
* @dev_priv: i915 device private
*
* Return: non-zero if the initialization failed.
*/
-int intel_engines_init(struct drm_i915_private *dev_priv)
+int intel_engines_init_early(struct drm_i915_private *dev_priv)
{
struct intel_device_info *device_info = mkwrite_device_info(dev_priv);
unsigned int ring_mask = INTEL_INFO(dev_priv)->ring_mask;
unsigned int mask = 0;
- int (*init)(struct intel_engine_cs *engine);
struct intel_engine_cs *engine;
enum intel_engine_id id;
unsigned int i;
- int ret;
+ int err;
WARN_ON(ring_mask == 0);
WARN_ON(ring_mask &
if (!HAS_ENGINE(dev_priv, i))
continue;
- if (i915.enable_execlists)
- init = intel_engines[i].init_execlists;
- else
- init = intel_engines[i].init_legacy;
-
- if (!init)
- continue;
-
- ret = intel_engine_setup(dev_priv, i);
- if (ret)
- goto cleanup;
-
- ret = init(dev_priv->engine[i]);
- if (ret)
+ err = intel_engine_setup(dev_priv, i);
+ if (err)
goto cleanup;
mask |= ENGINE_MASK(i);
return 0;
cleanup:
+ for_each_engine(engine, dev_priv, id)
+ kfree(engine);
+ return err;
+}
+
+/**
+ * intel_engines_init() - allocate, populate and init the Engine Command Streamers
+ * @dev_priv: i915 device private
+ *
+ * Return: non-zero if the initialization failed.
+ */
+int intel_engines_init(struct drm_i915_private *dev_priv)
+{
+ struct intel_device_info *device_info = mkwrite_device_info(dev_priv);
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id, err_id;
+ unsigned int mask = 0;
+ int err = 0;
+
for_each_engine(engine, dev_priv, id) {
+ int (*init)(struct intel_engine_cs *engine);
+
if (i915.enable_execlists)
- intel_logical_ring_cleanup(engine);
+ init = intel_engines[id].init_execlists;
else
- intel_engine_cleanup(engine);
+ init = intel_engines[id].init_legacy;
+ if (!init) {
+ kfree(engine);
+ dev_priv->engine[id] = NULL;
+ continue;
+ }
+
+ err = init(engine);
+ if (err) {
+ err_id = id;
+ goto cleanup;
+ }
+
+ GEM_BUG_ON(!engine->submit_request);
+ mask |= ENGINE_MASK(id);
}
- return ret;
+ /*
+ * Catch failures to update intel_engines table when the new engines
+ * are added to the driver by a warning and disabling the forgotten
+ * engines.
+ */
+ if (WARN_ON(mask != INTEL_INFO(dev_priv)->ring_mask))
+ device_info->ring_mask = mask;
+
+ device_info->num_rings = hweight32(mask);
+
+ return 0;
+
+cleanup:
+ for_each_engine(engine, dev_priv, id) {
+ if (id >= err_id)
+ kfree(engine);
+ else
+ dev_priv->gt.cleanup_engine(engine);
+ }
+ return err;
}
void intel_engine_init_global_seqno(struct intel_engine_cs *engine, u32 seqno)
}
intel_write_status_page(engine, I915_GEM_HWS_INDEX, seqno);
- if (engine->irq_seqno_barrier)
- engine->irq_seqno_barrier(engine);
+ clear_bit(ENGINE_IRQ_BREADCRUMB, &engine->irq_posted);
GEM_BUG_ON(i915_gem_active_isset(&engine->timeline->last_request));
- engine->timeline->last_submitted_seqno = seqno;
-
engine->hangcheck.seqno = seqno;
/* After manually advancing the seqno, fake the interrupt in case
{
int ret;
+ engine->set_default_submission(engine);
+
/* We may need to do things with the shrinker which
* require us to immediately switch back to the default
* context. This can cause a problem as pinning the
break;
}
}
+
+static int wa_add(struct drm_i915_private *dev_priv,
+ i915_reg_t addr,
+ const u32 mask, const u32 val)
+{
+ const u32 idx = dev_priv->workarounds.count;
+
+ if (WARN_ON(idx >= I915_MAX_WA_REGS))
+ return -ENOSPC;
+
+ dev_priv->workarounds.reg[idx].addr = addr;
+ dev_priv->workarounds.reg[idx].value = val;
+ dev_priv->workarounds.reg[idx].mask = mask;
+
+ dev_priv->workarounds.count++;
+
+ return 0;
+}
+
+#define WA_REG(addr, mask, val) do { \
+ const int r = wa_add(dev_priv, (addr), (mask), (val)); \
+ if (r) \
+ return r; \
+ } while (0)
+
+#define WA_SET_BIT_MASKED(addr, mask) \
+ WA_REG(addr, (mask), _MASKED_BIT_ENABLE(mask))
+
+#define WA_CLR_BIT_MASKED(addr, mask) \
+ WA_REG(addr, (mask), _MASKED_BIT_DISABLE(mask))
+
+#define WA_SET_FIELD_MASKED(addr, mask, value) \
+ WA_REG(addr, mask, _MASKED_FIELD(mask, value))
+
+#define WA_SET_BIT(addr, mask) WA_REG(addr, mask, I915_READ(addr) | (mask))
+#define WA_CLR_BIT(addr, mask) WA_REG(addr, mask, I915_READ(addr) & ~(mask))
+
+#define WA_WRITE(addr, val) WA_REG(addr, 0xffffffff, val)
+
+static int wa_ring_whitelist_reg(struct intel_engine_cs *engine,
+ i915_reg_t reg)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+ struct i915_workarounds *wa = &dev_priv->workarounds;
+ const uint32_t index = wa->hw_whitelist_count[engine->id];
+
+ if (WARN_ON(index >= RING_MAX_NONPRIV_SLOTS))
+ return -EINVAL;
+
+ WA_WRITE(RING_FORCE_TO_NONPRIV(engine->mmio_base, index),
+ i915_mmio_reg_offset(reg));
+ wa->hw_whitelist_count[engine->id]++;
+
+ return 0;
+}
+
+static int gen8_init_workarounds(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+
+ WA_SET_BIT_MASKED(INSTPM, INSTPM_FORCE_ORDERING);
+
+ /* WaDisableAsyncFlipPerfMode:bdw,chv */
+ WA_SET_BIT_MASKED(MI_MODE, ASYNC_FLIP_PERF_DISABLE);
+
+ /* WaDisablePartialInstShootdown:bdw,chv */
+ WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
+ PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
+
+ /* Use Force Non-Coherent whenever executing a 3D context. This is a
+ * workaround for for a possible hang in the unlikely event a TLB
+ * invalidation occurs during a PSD flush.
+ */
+ /* WaForceEnableNonCoherent:bdw,chv */
+ /* WaHdcDisableFetchWhenMasked:bdw,chv */
+ WA_SET_BIT_MASKED(HDC_CHICKEN0,
+ HDC_DONOT_FETCH_MEM_WHEN_MASKED |
+ HDC_FORCE_NON_COHERENT);
+
+ /* From the Haswell PRM, Command Reference: Registers, CACHE_MODE_0:
+ * "The Hierarchical Z RAW Stall Optimization allows non-overlapping
+ * polygons in the same 8x4 pixel/sample area to be processed without
+ * stalling waiting for the earlier ones to write to Hierarchical Z
+ * buffer."
+ *
+ * This optimization is off by default for BDW and CHV; turn it on.
+ */
+ WA_CLR_BIT_MASKED(CACHE_MODE_0_GEN7, HIZ_RAW_STALL_OPT_DISABLE);
+
+ /* Wa4x4STCOptimizationDisable:bdw,chv */
+ WA_SET_BIT_MASKED(CACHE_MODE_1, GEN8_4x4_STC_OPTIMIZATION_DISABLE);
+
+ /*
+ * BSpec recommends 8x4 when MSAA is used,
+ * however in practice 16x4 seems fastest.
+ *
+ * Note that PS/WM thread counts depend on the WIZ hashing
+ * disable bit, which we don't touch here, but it's good
+ * to keep in mind (see 3DSTATE_PS and 3DSTATE_WM).
+ */
+ WA_SET_FIELD_MASKED(GEN7_GT_MODE,
+ GEN6_WIZ_HASHING_MASK,
+ GEN6_WIZ_HASHING_16x4);
+
+ return 0;
+}
+
+static int bdw_init_workarounds(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+ int ret;
+
+ ret = gen8_init_workarounds(engine);
+ if (ret)
+ return ret;
+
+ /* WaDisableThreadStallDopClockGating:bdw (pre-production) */
+ WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
+
+ /* WaDisableDopClockGating:bdw
+ *
+ * Also see the related UCGTCL1 write in broadwell_init_clock_gating()
+ * to disable EUTC clock gating.
+ */
+ WA_SET_BIT_MASKED(GEN7_ROW_CHICKEN2,
+ DOP_CLOCK_GATING_DISABLE);
+
+ WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
+ GEN8_SAMPLER_POWER_BYPASS_DIS);
+
+ WA_SET_BIT_MASKED(HDC_CHICKEN0,
+ /* WaForceContextSaveRestoreNonCoherent:bdw */
+ HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT |
+ /* WaDisableFenceDestinationToSLM:bdw (pre-prod) */
+ (IS_BDW_GT3(dev_priv) ? HDC_FENCE_DEST_SLM_DISABLE : 0));
+
+ return 0;
+}
+
+static int chv_init_workarounds(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+ int ret;
+
+ ret = gen8_init_workarounds(engine);
+ if (ret)
+ return ret;
+
+ /* WaDisableThreadStallDopClockGating:chv */
+ WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
+
+ /* Improve HiZ throughput on CHV. */
+ WA_SET_BIT_MASKED(HIZ_CHICKEN, CHV_HZ_8X8_MODE_IN_1X);
+
+ return 0;
+}
+
+static int gen9_init_workarounds(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+ int ret;
+
+ /* WaConextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk */
+ I915_WRITE(GEN9_CSFE_CHICKEN1_RCS, _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));
+
+ /* WaEnableLbsSlaRetryTimerDecrement:skl,bxt,kbl,glk */
+ I915_WRITE(BDW_SCRATCH1, I915_READ(BDW_SCRATCH1) |
+ GEN9_LBS_SLA_RETRY_TIMER_DECREMENT_ENABLE);
+
+ /* WaDisableKillLogic:bxt,skl,kbl */
+ I915_WRITE(GAM_ECOCHK, I915_READ(GAM_ECOCHK) |
+ ECOCHK_DIS_TLB);
+
+ /* WaClearFlowControlGpgpuContextSave:skl,bxt,kbl,glk */
+ /* WaDisablePartialInstShootdown:skl,bxt,kbl,glk */
+ WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
+ FLOW_CONTROL_ENABLE |
+ PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
+
+ /* Syncing dependencies between camera and graphics:skl,bxt,kbl */
+ WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
+ GEN9_DISABLE_OCL_OOB_SUPPRESS_LOGIC);
+
+ /* WaDisableDgMirrorFixInHalfSliceChicken5:bxt */
+ if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1))
+ WA_CLR_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN5,
+ GEN9_DG_MIRROR_FIX_ENABLE);
+
+ /* WaSetDisablePixMaskCammingAndRhwoInCommonSliceChicken:bxt */
+ if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1)) {
+ WA_SET_BIT_MASKED(GEN7_COMMON_SLICE_CHICKEN1,
+ GEN9_RHWO_OPTIMIZATION_DISABLE);
+ /*
+ * WA also requires GEN9_SLICE_COMMON_ECO_CHICKEN0[14:14] to be set
+ * but we do that in per ctx batchbuffer as there is an issue
+ * with this register not getting restored on ctx restore
+ */
+ }
+
+ /* WaEnableSamplerGPGPUPreemptionSupport:skl,bxt,kbl */
+ WA_SET_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN7,
+ GEN9_ENABLE_GPGPU_PREEMPTION);
+
+ /* Wa4x4STCOptimizationDisable:skl,bxt,kbl,glk */
+ /* WaDisablePartialResolveInVc:skl,bxt,kbl */
+ WA_SET_BIT_MASKED(CACHE_MODE_1, (GEN8_4x4_STC_OPTIMIZATION_DISABLE |
+ GEN9_PARTIAL_RESOLVE_IN_VC_DISABLE));
+
+ /* WaCcsTlbPrefetchDisable:skl,bxt,kbl,glk */
+ WA_CLR_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN5,
+ GEN9_CCS_TLB_PREFETCH_ENABLE);
+
+ /* WaDisableMaskBasedCammingInRCC:bxt */
+ if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1))
+ WA_SET_BIT_MASKED(SLICE_ECO_CHICKEN0,
+ PIXEL_MASK_CAMMING_DISABLE);
+
+ /* WaForceContextSaveRestoreNonCoherent:skl,bxt,kbl */
+ WA_SET_BIT_MASKED(HDC_CHICKEN0,
+ HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT |
+ HDC_FORCE_CSR_NON_COHERENT_OVR_DISABLE);
+
+ /* WaForceEnableNonCoherent and WaDisableHDCInvalidation are
+ * both tied to WaForceContextSaveRestoreNonCoherent
+ * in some hsds for skl. We keep the tie for all gen9. The
+ * documentation is a bit hazy and so we want to get common behaviour,
+ * even though there is no clear evidence we would need both on kbl/bxt.
+ * This area has been source of system hangs so we play it safe
+ * and mimic the skl regardless of what bspec says.
+ *
+ * Use Force Non-Coherent whenever executing a 3D context. This
+ * is a workaround for a possible hang in the unlikely event
+ * a TLB invalidation occurs during a PSD flush.
+ */
+
+ /* WaForceEnableNonCoherent:skl,bxt,kbl */
+ WA_SET_BIT_MASKED(HDC_CHICKEN0,
+ HDC_FORCE_NON_COHERENT);
+
+ /* WaDisableHDCInvalidation:skl,bxt,kbl */
+ I915_WRITE(GAM_ECOCHK, I915_READ(GAM_ECOCHK) |
+ BDW_DISABLE_HDC_INVALIDATION);
+
+ /* WaDisableSamplerPowerBypassForSOPingPong:skl,bxt,kbl */
+ if (IS_SKYLAKE(dev_priv) ||
+ IS_KABYLAKE(dev_priv) ||
+ IS_BXT_REVID(dev_priv, 0, BXT_REVID_B0))
+ WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
+ GEN8_SAMPLER_POWER_BYPASS_DIS);
+
+ /* WaDisableSTUnitPowerOptimization:skl,bxt,kbl,glk */
+ WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN2, GEN8_ST_PO_DISABLE);
+
+ /* WaOCLCoherentLineFlush:skl,bxt,kbl */
+ I915_WRITE(GEN8_L3SQCREG4, (I915_READ(GEN8_L3SQCREG4) |
+ GEN8_LQSC_FLUSH_COHERENT_LINES));
+
+ /* WaVFEStateAfterPipeControlwithMediaStateClear:skl,bxt,glk */
+ ret = wa_ring_whitelist_reg(engine, GEN9_CTX_PREEMPT_REG);
+ if (ret)
+ return ret;
+
+ /* WaEnablePreemptionGranularityControlByUMD:skl,bxt,kbl */
+ ret= wa_ring_whitelist_reg(engine, GEN8_CS_CHICKEN1);
+ if (ret)
+ return ret;
+
+ /* WaAllowUMDToModifyHDCChicken1:skl,bxt,kbl,glk */
+ ret = wa_ring_whitelist_reg(engine, GEN8_HDC_CHICKEN1);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int skl_tune_iz_hashing(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+ u8 vals[3] = { 0, 0, 0 };
+ unsigned int i;
+
+ for (i = 0; i < 3; i++) {
+ u8 ss;
+
+ /*
+ * Only consider slices where one, and only one, subslice has 7
+ * EUs
+ */
+ if (!is_power_of_2(INTEL_INFO(dev_priv)->sseu.subslice_7eu[i]))
+ continue;
+
+ /*
+ * subslice_7eu[i] != 0 (because of the check above) and
+ * ss_max == 4 (maximum number of subslices possible per slice)
+ *
+ * -> 0 <= ss <= 3;
+ */
+ ss = ffs(INTEL_INFO(dev_priv)->sseu.subslice_7eu[i]) - 1;
+ vals[i] = 3 - ss;
+ }
+
+ if (vals[0] == 0 && vals[1] == 0 && vals[2] == 0)
+ return 0;
+
+ /* Tune IZ hashing. See intel_device_info_runtime_init() */
+ WA_SET_FIELD_MASKED(GEN7_GT_MODE,
+ GEN9_IZ_HASHING_MASK(2) |
+ GEN9_IZ_HASHING_MASK(1) |
+ GEN9_IZ_HASHING_MASK(0),
+ GEN9_IZ_HASHING(2, vals[2]) |
+ GEN9_IZ_HASHING(1, vals[1]) |
+ GEN9_IZ_HASHING(0, vals[0]));
+
+ return 0;
+}
+
+static int skl_init_workarounds(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+ int ret;
+
+ ret = gen9_init_workarounds(engine);
+ if (ret)
+ return ret;
+
+ /*
+ * Actual WA is to disable percontext preemption granularity control
+ * until D0 which is the default case so this is equivalent to
+ * !WaDisablePerCtxtPreemptionGranularityControl:skl
+ */
+ I915_WRITE(GEN7_FF_SLICE_CS_CHICKEN1,
+ _MASKED_BIT_ENABLE(GEN9_FFSC_PERCTX_PREEMPT_CTRL));
+
+ /* WaEnableGapsTsvCreditFix:skl */
+ I915_WRITE(GEN8_GARBCNTL, (I915_READ(GEN8_GARBCNTL) |
+ GEN9_GAPS_TSV_CREDIT_DISABLE));
+
+ /* WaDisableGafsUnitClkGating:skl */
+ WA_SET_BIT(GEN7_UCGCTL4, GEN8_EU_GAUNIT_CLOCK_GATE_DISABLE);
+
+ /* WaInPlaceDecompressionHang:skl */
+ if (IS_SKL_REVID(dev_priv, SKL_REVID_H0, REVID_FOREVER))
+ WA_SET_BIT(GEN9_GAMT_ECO_REG_RW_IA,
+ GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
+
+ /* WaDisableLSQCROPERFforOCL:skl */
+ ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
+ if (ret)
+ return ret;
+
+ return skl_tune_iz_hashing(engine);
+}
+
+static int bxt_init_workarounds(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+ int ret;
+
+ ret = gen9_init_workarounds(engine);
+ if (ret)
+ return ret;
+
+ /* WaStoreMultiplePTEenable:bxt */
+ /* This is a requirement according to Hardware specification */
+ if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1))
+ I915_WRITE(TILECTL, I915_READ(TILECTL) | TILECTL_TLBPF);
+
+ /* WaSetClckGatingDisableMedia:bxt */
+ if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1)) {
+ I915_WRITE(GEN7_MISCCPCTL, (I915_READ(GEN7_MISCCPCTL) &
+ ~GEN8_DOP_CLOCK_GATE_MEDIA_ENABLE));
+ }
+
+ /* WaDisableThreadStallDopClockGating:bxt */
+ WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
+ STALL_DOP_GATING_DISABLE);
+
+ /* WaDisablePooledEuLoadBalancingFix:bxt */
+ if (IS_BXT_REVID(dev_priv, BXT_REVID_B0, REVID_FOREVER)) {
+ WA_SET_BIT_MASKED(FF_SLICE_CS_CHICKEN2,
+ GEN9_POOLED_EU_LOAD_BALANCING_FIX_DISABLE);
+ }
+
+ /* WaDisableSbeCacheDispatchPortSharing:bxt */
+ if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_B0)) {
+ WA_SET_BIT_MASKED(
+ GEN7_HALF_SLICE_CHICKEN1,
+ GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
+ }
+
+ /* WaDisableObjectLevelPreemptionForTrifanOrPolygon:bxt */
+ /* WaDisableObjectLevelPreemptionForInstancedDraw:bxt */
+ /* WaDisableObjectLevelPreemtionForInstanceId:bxt */
+ /* WaDisableLSQCROPERFforOCL:bxt */
+ if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1)) {
+ ret = wa_ring_whitelist_reg(engine, GEN9_CS_DEBUG_MODE1);
+ if (ret)
+ return ret;
+
+ ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
+ if (ret)
+ return ret;
+ }
+
+ /* WaProgramL3SqcReg1DefaultForPerf:bxt */
+ if (IS_BXT_REVID(dev_priv, BXT_REVID_B0, REVID_FOREVER))
+ I915_WRITE(GEN8_L3SQCREG1, L3_GENERAL_PRIO_CREDITS(62) |
+ L3_HIGH_PRIO_CREDITS(2));
+
+ /* WaToEnableHwFixForPushConstHWBug:bxt */
+ if (IS_BXT_REVID(dev_priv, BXT_REVID_C0, REVID_FOREVER))
+ WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
+ GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
+
+ /* WaInPlaceDecompressionHang:bxt */
+ if (IS_BXT_REVID(dev_priv, BXT_REVID_C0, REVID_FOREVER))
+ WA_SET_BIT(GEN9_GAMT_ECO_REG_RW_IA,
+ GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
+
+ return 0;
+}
+
+static int kbl_init_workarounds(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+ int ret;
+
+ ret = gen9_init_workarounds(engine);
+ if (ret)
+ return ret;
+
+ /* WaEnableGapsTsvCreditFix:kbl */
+ I915_WRITE(GEN8_GARBCNTL, (I915_READ(GEN8_GARBCNTL) |
+ GEN9_GAPS_TSV_CREDIT_DISABLE));
+
+ /* WaDisableDynamicCreditSharing:kbl */
+ if (IS_KBL_REVID(dev_priv, 0, KBL_REVID_B0))
+ WA_SET_BIT(GAMT_CHKN_BIT_REG,
+ GAMT_CHKN_DISABLE_DYNAMIC_CREDIT_SHARING);
+
+ /* WaDisableFenceDestinationToSLM:kbl (pre-prod) */
+ if (IS_KBL_REVID(dev_priv, KBL_REVID_A0, KBL_REVID_A0))
+ WA_SET_BIT_MASKED(HDC_CHICKEN0,
+ HDC_FENCE_DEST_SLM_DISABLE);
+
+ /* WaToEnableHwFixForPushConstHWBug:kbl */
+ if (IS_KBL_REVID(dev_priv, KBL_REVID_C0, REVID_FOREVER))
+ WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
+ GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
+
+ /* WaDisableGafsUnitClkGating:kbl */
+ WA_SET_BIT(GEN7_UCGCTL4, GEN8_EU_GAUNIT_CLOCK_GATE_DISABLE);
+
+ /* WaDisableSbeCacheDispatchPortSharing:kbl */
+ WA_SET_BIT_MASKED(
+ GEN7_HALF_SLICE_CHICKEN1,
+ GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
+
+ /* WaInPlaceDecompressionHang:kbl */
+ WA_SET_BIT(GEN9_GAMT_ECO_REG_RW_IA,
+ GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
+
+ /* WaDisableLSQCROPERFforOCL:kbl */
+ ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int glk_init_workarounds(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+ int ret;
+
+ ret = gen9_init_workarounds(engine);
+ if (ret)
+ return ret;
+
+ /* WaToEnableHwFixForPushConstHWBug:glk */
+ WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
+ GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
+
+ return 0;
+}
+
+int init_workarounds_ring(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+ int err;
+
+ WARN_ON(engine->id != RCS);
+
+ dev_priv->workarounds.count = 0;
+ dev_priv->workarounds.hw_whitelist_count[engine->id] = 0;
+
+ if (IS_BROADWELL(dev_priv))
+ err = bdw_init_workarounds(engine);
+ else if (IS_CHERRYVIEW(dev_priv))
+ err = chv_init_workarounds(engine);
+ else if (IS_SKYLAKE(dev_priv))
+ err = skl_init_workarounds(engine);
+ else if (IS_BROXTON(dev_priv))
+ err = bxt_init_workarounds(engine);
+ else if (IS_KABYLAKE(dev_priv))
+ err = kbl_init_workarounds(engine);
+ else if (IS_GEMINILAKE(dev_priv))
+ err = glk_init_workarounds(engine);
+ else
+ err = 0;
+ if (err)
+ return err;
+
+ DRM_DEBUG_DRIVER("%s: Number of context specific w/a: %d\n",
+ engine->name, dev_priv->workarounds.count);
+ return 0;
+}
+
+int intel_ring_workarounds_emit(struct drm_i915_gem_request *req)
+{
+ struct i915_workarounds *w = &req->i915->workarounds;
+ u32 *cs;
+ int ret, i;
+
+ if (w->count == 0)
+ return 0;
+
+ ret = req->engine->emit_flush(req, EMIT_BARRIER);
+ if (ret)
+ return ret;
+
+ cs = intel_ring_begin(req, (w->count * 2 + 2));
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
+
+ *cs++ = MI_LOAD_REGISTER_IMM(w->count);
+ for (i = 0; i < w->count; i++) {
+ *cs++ = i915_mmio_reg_offset(w->reg[i].addr);
+ *cs++ = w->reg[i].value;
+ }
+ *cs++ = MI_NOOP;
+
+ intel_ring_advance(req, cs);
+
+ ret = req->engine->emit_flush(req, EMIT_BARRIER);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+/**
+ * intel_engine_is_idle() - Report if the engine has finished process all work
+ * @engine: the intel_engine_cs
+ *
+ * Return true if there are no requests pending, nothing left to be submitted
+ * to hardware, and that the engine is idle.
+ */
+bool intel_engine_is_idle(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+
+ /* Any inflight/incomplete requests? */
+ if (!i915_seqno_passed(intel_engine_get_seqno(engine),
+ intel_engine_last_submit(engine)))
+ return false;
+
+ /* Interrupt/tasklet pending? */
+ if (test_bit(ENGINE_IRQ_EXECLIST, &engine->irq_posted))
+ return false;
+
+ /* Both ports drained, no more ELSP submission? */
+ if (engine->execlist_port[0].request)
+ return false;
+
+ /* Ring stopped? */
+ if (INTEL_GEN(dev_priv) > 2 && !(I915_READ_MODE(engine) & MODE_IDLE))
+ return false;
+
+ return true;
+}
+
+bool intel_engines_are_idle(struct drm_i915_private *dev_priv)
+{
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+
+ for_each_engine(engine, dev_priv, id) {
+ if (!intel_engine_is_idle(engine))
+ return false;
+ }
+
+ return true;
+}
+
+void intel_engines_reset_default_submission(struct drm_i915_private *i915)
+{
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+
+ for_each_engine(engine, i915, id)
+ engine->set_default_submission(engine);
+}
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/mock_engine.c"
+#endif
* reserved range size, so it always assumes the maximum (8mb) is used.
* If we enable FBC using a CFB on that memory range we'll get FIFO
* underruns, even if that range is not reserved by the BIOS. */
- if (IS_BROADWELL(dev_priv) ||
- IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ if (IS_BROADWELL(dev_priv) || IS_GEN9_BC(dev_priv))
end = ggtt->stolen_size - 8 * 1024 * 1024;
else
end = U64_MAX;
kfree(compressed_llb);
i915_gem_stolen_remove_node(dev_priv, &fbc->compressed_fb);
err_llb:
- pr_info_once("drm: not enough stolen space for compressed buffer (need %d more bytes), disabling. Hint: you may be able to increase stolen memory size in the BIOS to avoid this.\n", size);
+ if (drm_mm_initialized(&dev_priv->mm.stolen))
+ pr_info_once("drm: not enough stolen space for compressed buffer (need %d more bytes), disabling. Hint: you may be able to increase stolen memory size in the BIOS to avoid this.\n", size);
return -ENOSPC;
}
cache->crtc.mode_flags = crtc_state->base.adjusted_mode.flags;
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
- cache->crtc.hsw_bdw_pixel_rate =
- ilk_pipe_pixel_rate(crtc_state);
+ cache->crtc.hsw_bdw_pixel_rate = crtc_state->pixel_rate;
cache->plane.rotation = plane_state->base.rotation;
cache->plane.src_w = drm_rect_width(&plane_state->base.src) >> 16;
/* WaFbcExceedCdClockThreshold:hsw,bdw */
if ((IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) &&
- cache->crtc.hsw_bdw_pixel_rate >= dev_priv->cdclk_freq * 95 / 100) {
+ cache->crtc.hsw_bdw_pixel_rate >= dev_priv->cdclk.hw.cdclk * 95 / 100) {
fbc->no_fbc_reason = "pixel rate is too big";
return false;
}
* plane. We could go for fancier schemes such as checking the plane
* size, but this would just affect the few platforms that don't tie FBC
* to pipe or plane A. */
- for_each_plane_in_state(state, plane, plane_state, i) {
+ for_each_new_plane_in_state(state, plane, plane_state, i) {
struct intel_plane_state *intel_plane_state =
to_intel_plane_state(plane_state);
struct intel_crtc_state *intel_crtc_state;
#include <drm/i915_drm.h>
#include "i915_drv.h"
+static void intel_fbdev_invalidate(struct intel_fbdev *ifbdev)
+{
+ struct drm_i915_gem_object *obj = ifbdev->fb->obj;
+ unsigned int origin = ifbdev->vma->fence ? ORIGIN_GTT : ORIGIN_CPU;
+
+ intel_fb_obj_invalidate(obj, origin);
+}
+
static int intel_fbdev_set_par(struct fb_info *info)
{
struct drm_fb_helper *fb_helper = info->par;
int ret;
ret = drm_fb_helper_set_par(info);
-
- if (ret == 0) {
- mutex_lock(&fb_helper->dev->struct_mutex);
- intel_fb_obj_invalidate(ifbdev->fb->obj, ORIGIN_GTT);
- mutex_unlock(&fb_helper->dev->struct_mutex);
- }
+ if (ret == 0)
+ intel_fbdev_invalidate(ifbdev);
return ret;
}
int ret;
ret = drm_fb_helper_blank(blank, info);
-
- if (ret == 0) {
- mutex_lock(&fb_helper->dev->struct_mutex);
- intel_fb_obj_invalidate(ifbdev->fb->obj, ORIGIN_GTT);
- mutex_unlock(&fb_helper->dev->struct_mutex);
- }
+ if (ret == 0)
+ intel_fbdev_invalidate(ifbdev);
return ret;
}
struct drm_fb_helper *fb_helper = info->par;
struct intel_fbdev *ifbdev =
container_of(fb_helper, struct intel_fbdev, helper);
-
int ret;
- ret = drm_fb_helper_pan_display(var, info);
- if (ret == 0) {
- mutex_lock(&fb_helper->dev->struct_mutex);
- intel_fb_obj_invalidate(ifbdev->fb->obj, ORIGIN_GTT);
- mutex_unlock(&fb_helper->dev->struct_mutex);
- }
+ ret = drm_fb_helper_pan_display(var, info);
+ if (ret == 0)
+ intel_fbdev_invalidate(ifbdev);
return ret;
}
struct drm_i915_private *dev_priv = to_i915(dev);
struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct drm_mode_fb_cmd2 mode_cmd = {};
- struct drm_i915_gem_object *obj = NULL;
+ struct drm_i915_gem_object *obj;
int size, ret;
/* we don't do packed 24bpp */
mode_cmd.pixel_format = drm_mode_legacy_fb_format(sizes->surface_bpp,
sizes->surface_depth);
- mutex_lock(&dev->struct_mutex);
-
size = mode_cmd.pitches[0] * mode_cmd.height;
size = PAGE_ALIGN(size);
/* If the FB is too big, just don't use it since fbdev is not very
* important and we should probably use that space with FBC or other
* features. */
+ obj = NULL;
if (size * 2 < ggtt->stolen_usable_size)
obj = i915_gem_object_create_stolen(dev_priv, size);
if (obj == NULL)
if (IS_ERR(obj)) {
DRM_ERROR("failed to allocate framebuffer\n");
ret = PTR_ERR(obj);
- goto out;
+ goto err;
}
- fb = __intel_framebuffer_create(dev, &mode_cmd, obj);
+ fb = intel_framebuffer_create(obj, &mode_cmd);
if (IS_ERR(fb)) {
- i915_gem_object_put(obj);
ret = PTR_ERR(fb);
- goto out;
+ goto err_obj;
}
- mutex_unlock(&dev->struct_mutex);
-
ifbdev->fb = to_intel_framebuffer(fb);
return 0;
-out:
- mutex_unlock(&dev->struct_mutex);
+err_obj:
+ i915_gem_object_put(obj);
+err:
return ret;
}
if (IS_ERR(vaddr)) {
DRM_ERROR("Failed to remap framebuffer into virtual memory\n");
ret = PTR_ERR(vaddr);
- goto out_destroy_fbi;
+ goto out_unpin;
}
info->screen_base = vaddr;
info->screen_size = vma->node.size;
vga_switcheroo_client_fb_set(pdev, info);
return 0;
-out_destroy_fbi:
- drm_fb_helper_release_fbi(helper);
out_unpin:
intel_unpin_fb_vma(vma);
out_unlock:
return false;
memcpy(save_enabled, enabled, count);
- mask = BIT(count) - 1;
+ mask = GENMASK(count - 1, 0);
conn_configured = 0;
retry:
conn_seq = conn_configured;
*/
drm_fb_helper_unregister_fbi(&ifbdev->helper);
- drm_fb_helper_release_fbi(&ifbdev->helper);
drm_fb_helper_fini(&ifbdev->helper);
}
cur_size = intel_crtc->config->base.adjusted_mode.crtc_vdisplay;
- cur_size = intel_fb_align_height(dev, cur_size,
- fb->base.format->format,
- fb->base.modifier);
+ cur_size = intel_fb_align_height(&fb->base, 0, cur_size);
cur_size *= fb->base.pitches[0];
DRM_DEBUG_KMS("pipe %c area: %dx%d, bpp: %d, size: %d\n",
pipe_name(intel_crtc->pipe),
if (!ifbdev->fb)
return;
- if (drm_fb_helper_restore_fbdev_mode_unlocked(&ifbdev->helper)) {
- DRM_DEBUG("failed to restore crtc mode\n");
- } else {
- mutex_lock(&dev->struct_mutex);
- intel_fb_obj_invalidate(ifbdev->fb->obj, ORIGIN_GTT);
- mutex_unlock(&dev->struct_mutex);
- }
+ if (drm_fb_helper_restore_fbdev_mode_unlocked(&ifbdev->helper) == 0)
+ intel_fbdev_invalidate(ifbdev);
}
struct intel_crtc *crtc;
enum pipe pipe;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
for_each_pipe(dev_priv, pipe) {
crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
enum pipe pipe;
struct intel_crtc *crtc;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
for_each_pipe(dev_priv, pipe) {
crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
i915_reg_t reg = PIPESTAT(crtc->pipe);
u32 pipestat = I915_READ(reg) & 0xffff0000;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
if ((pipestat & PIPE_FIFO_UNDERRUN_STATUS) == 0)
return;
I915_WRITE(reg, pipestat | PIPE_FIFO_UNDERRUN_STATUS);
POSTING_READ(reg);
+ trace_intel_cpu_fifo_underrun(dev_priv, crtc->pipe);
DRM_ERROR("pipe %c underrun\n", pipe_name(crtc->pipe));
}
i915_reg_t reg = PIPESTAT(pipe);
u32 pipestat = I915_READ(reg) & 0xffff0000;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
if (enable) {
I915_WRITE(reg, pipestat | PIPE_FIFO_UNDERRUN_STATUS);
enum pipe pipe = crtc->pipe;
uint32_t err_int = I915_READ(GEN7_ERR_INT);
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
if ((err_int & ERR_INT_FIFO_UNDERRUN(pipe)) == 0)
return;
I915_WRITE(GEN7_ERR_INT, ERR_INT_FIFO_UNDERRUN(pipe));
POSTING_READ(GEN7_ERR_INT);
+ trace_intel_cpu_fifo_underrun(dev_priv, pipe);
DRM_ERROR("fifo underrun on pipe %c\n", pipe_name(pipe));
}
enum transcoder pch_transcoder = (enum transcoder) crtc->pipe;
uint32_t serr_int = I915_READ(SERR_INT);
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
if ((serr_int & SERR_INT_TRANS_FIFO_UNDERRUN(pch_transcoder)) == 0)
return;
I915_WRITE(SERR_INT, SERR_INT_TRANS_FIFO_UNDERRUN(pch_transcoder));
POSTING_READ(SERR_INT);
+ trace_intel_pch_fifo_underrun(dev_priv, pch_transcoder);
DRM_ERROR("pch fifo underrun on pch transcoder %s\n",
transcoder_name(pch_transcoder));
}
struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
bool old;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
old = !crtc->cpu_fifo_underrun_disabled;
crtc->cpu_fifo_underrun_disabled = !enable;
crtc->cpu_fifo_underrun_disabled)
return;
- if (intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false))
+ if (intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false)) {
+ trace_intel_cpu_fifo_underrun(dev_priv, pipe);
DRM_ERROR("CPU pipe %c FIFO underrun\n",
pipe_name(pipe));
+ }
intel_fbc_handle_fifo_underrun_irq(dev_priv);
}
enum transcoder pch_transcoder)
{
if (intel_set_pch_fifo_underrun_reporting(dev_priv, pch_transcoder,
- false))
+ false)) {
+ trace_intel_pch_fifo_underrun(dev_priv, pch_transcoder);
DRM_ERROR("PCH transcoder %s FIFO underrun\n",
transcoder_name(pch_transcoder));
+ }
}
/**
}
void __intel_fb_obj_flush(struct drm_i915_gem_object *obj,
- bool retire,
enum fb_op_origin origin,
unsigned int frontbuffer_bits)
{
struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
- if (retire) {
+ if (origin == ORIGIN_CS) {
spin_lock(&dev_priv->fb_tracking.lock);
/* Filter out new bits since rendering started. */
frontbuffer_bits &= dev_priv->fb_tracking.busy_bits;
enum fb_op_origin origin,
unsigned int frontbuffer_bits);
void __intel_fb_obj_flush(struct drm_i915_gem_object *obj,
- bool retire,
enum fb_op_origin origin,
unsigned int frontbuffer_bits);
/**
* intel_fb_obj_flush - flush frontbuffer object
* @obj: GEM object to flush
- * @retire: set when retiring asynchronous rendering
* @origin: which operation caused the flush
*
* This function gets called every time rendering on the given object has
- * completed and frontbuffer caching can be started again. If @retire is true
- * then any delayed flushes will be unblocked.
+ * completed and frontbuffer caching can be started again.
*/
static inline void intel_fb_obj_flush(struct drm_i915_gem_object *obj,
- bool retire,
enum fb_op_origin origin)
{
unsigned int frontbuffer_bits;
if (!frontbuffer_bits)
return;
- __intel_fb_obj_flush(obj, retire, origin, frontbuffer_bits);
+ __intel_fb_obj_flush(obj, origin, frontbuffer_bits);
}
#endif /* __INTEL_FRONTBUFFER_H__ */
* Dave Gordon <david.s.gordon@intel.com>
* Alex Dai <yu.dai@intel.com>
*/
-#include <linux/firmware.h>
#include "i915_drv.h"
#include "intel_uc.h"
}
};
-static void guc_interrupts_release(struct drm_i915_private *dev_priv)
-{
- struct intel_engine_cs *engine;
- enum intel_engine_id id;
- int irqs;
-
- /* tell all command streamers NOT to forward interrupts or vblank to GuC */
- irqs = _MASKED_FIELD(GFX_FORWARD_VBLANK_MASK, GFX_FORWARD_VBLANK_NEVER);
- irqs |= _MASKED_BIT_DISABLE(GFX_INTERRUPT_STEERING);
- for_each_engine(engine, dev_priv, id)
- I915_WRITE(RING_MODE_GEN7(engine), irqs);
-
- /* route all GT interrupts to the host */
- I915_WRITE(GUC_BCS_RCS_IER, 0);
- I915_WRITE(GUC_VCS2_VCS1_IER, 0);
- I915_WRITE(GUC_WD_VECS_IER, 0);
-}
-
-static void guc_interrupts_capture(struct drm_i915_private *dev_priv)
-{
- struct intel_engine_cs *engine;
- enum intel_engine_id id;
- int irqs;
- u32 tmp;
-
- /* tell all command streamers to forward interrupts (but not vblank) to GuC */
- irqs = _MASKED_BIT_ENABLE(GFX_INTERRUPT_STEERING);
- for_each_engine(engine, dev_priv, id)
- I915_WRITE(RING_MODE_GEN7(engine), irqs);
-
- /* route USER_INTERRUPT to Host, all others are sent to GuC. */
- irqs = GT_RENDER_USER_INTERRUPT << GEN8_RCS_IRQ_SHIFT |
- GT_RENDER_USER_INTERRUPT << GEN8_BCS_IRQ_SHIFT;
- /* These three registers have the same bit definitions */
- I915_WRITE(GUC_BCS_RCS_IER, ~irqs);
- I915_WRITE(GUC_VCS2_VCS1_IER, ~irqs);
- I915_WRITE(GUC_WD_VECS_IER, ~irqs);
-
- /*
- * The REDIRECT_TO_GUC bit of the PMINTRMSK register directs all
- * (unmasked) PM interrupts to the GuC. All other bits of this
- * register *disable* generation of a specific interrupt.
- *
- * 'pm_intr_keep' indicates bits that are NOT to be set when
- * writing to the PM interrupt mask register, i.e. interrupts
- * that must not be disabled.
- *
- * If the GuC is handling these interrupts, then we must not let
- * the PM code disable ANY interrupt that the GuC is expecting.
- * So for each ENABLED (0) bit in this register, we must SET the
- * bit in pm_intr_keep so that it's left enabled for the GuC.
- *
- * OTOH the REDIRECT_TO_GUC bit is initially SET in pm_intr_keep
- * (so interrupts go to the DISPLAY unit at first); but here we
- * need to CLEAR that bit, which will result in the register bit
- * being left SET!
- */
- tmp = I915_READ(GEN6_PMINTRMSK);
- if (tmp & GEN8_PMINTR_REDIRECT_TO_GUC) {
- dev_priv->rps.pm_intr_keep |= ~tmp;
- dev_priv->rps.pm_intr_keep &= ~GEN8_PMINTR_REDIRECT_TO_GUC;
- }
-}
-
static u32 get_gttype(struct drm_i915_private *dev_priv)
{
/* XXX: GT type based on PCI device ID? field seems unused by fw */
return ret;
}
-static int guc_hw_reset(struct drm_i915_private *dev_priv)
-{
- int ret;
- u32 guc_status;
-
- ret = intel_guc_reset(dev_priv);
- if (ret) {
- DRM_ERROR("GuC reset failed, ret = %d\n", ret);
- return ret;
- }
-
- guc_status = I915_READ(GUC_STATUS);
- WARN(!(guc_status & GS_MIA_IN_RESET),
- "GuC status: 0x%x, MIA core expected to be in reset\n", guc_status);
-
- return ret;
-}
-
/**
- * intel_guc_setup() - finish preparing the GuC for activity
- * @dev_priv: i915 device private
+ * intel_guc_init_hw() - finish preparing the GuC for activity
+ * @guc: intel_guc structure
*
- * Called from gem_init_hw() during driver loading and also after a GPU reset.
+ * Called during driver loading and also after a GPU reset.
*
* The main action required here it to load the GuC uCode into the device.
* The firmware image should have already been fetched into memory by the
- * earlier call to intel_guc_init(), so here we need only check that worked,
- * and then transfer the image to the h/w.
+ * earlier call to intel_guc_init(), so here we need only check that
+ * worked, and then transfer the image to the h/w.
*
* Return: non-zero code on error
*/
-int intel_guc_setup(struct drm_i915_private *dev_priv)
+int intel_guc_init_hw(struct intel_guc *guc)
{
- struct intel_uc_fw *guc_fw = &dev_priv->guc.fw;
- const char *fw_path = guc_fw->path;
- int retries, ret, err;
+ struct drm_i915_private *dev_priv = guc_to_i915(guc);
+ const char *fw_path = guc->fw.path;
+ int ret;
DRM_DEBUG_DRIVER("GuC fw status: path %s, fetch %s, load %s\n",
fw_path,
- intel_uc_fw_status_repr(guc_fw->fetch_status),
- intel_uc_fw_status_repr(guc_fw->load_status));
-
- /* Loading forbidden, or no firmware to load? */
- if (!i915.enable_guc_loading) {
- err = 0;
- goto fail;
- } else if (fw_path == NULL) {
- /* Device is known to have no uCode (e.g. no GuC) */
- err = -ENXIO;
- goto fail;
- } else if (*fw_path == '\0') {
- /* Device has a GuC but we don't know what f/w to load? */
- WARN(1, "No GuC firmware known for this platform!\n");
- err = -ENODEV;
- goto fail;
- }
+ intel_uc_fw_status_repr(guc->fw.fetch_status),
+ intel_uc_fw_status_repr(guc->fw.load_status));
- /* Fetch failed, or already fetched but failed to load? */
- if (guc_fw->fetch_status != INTEL_UC_FIRMWARE_SUCCESS) {
- err = -EIO;
- goto fail;
- } else if (guc_fw->load_status == INTEL_UC_FIRMWARE_FAIL) {
- err = -ENOEXEC;
- goto fail;
- }
+ if (guc->fw.fetch_status != INTEL_UC_FIRMWARE_SUCCESS)
+ return -EIO;
- guc_interrupts_release(dev_priv);
- gen9_reset_guc_interrupts(dev_priv);
-
- /* We need to notify the guc whenever we change the GGTT */
- i915_ggtt_enable_guc(dev_priv);
-
- guc_fw->load_status = INTEL_UC_FIRMWARE_PENDING;
+ guc->fw.load_status = INTEL_UC_FIRMWARE_PENDING;
DRM_DEBUG_DRIVER("GuC fw status: fetch %s, load %s\n",
- intel_uc_fw_status_repr(guc_fw->fetch_status),
- intel_uc_fw_status_repr(guc_fw->load_status));
+ intel_uc_fw_status_repr(guc->fw.fetch_status),
+ intel_uc_fw_status_repr(guc->fw.load_status));
- err = i915_guc_submission_init(dev_priv);
- if (err)
- goto fail;
+ ret = guc_ucode_xfer(dev_priv);
- /*
- * WaEnableuKernelHeaderValidFix:skl,bxt
- * For BXT, this is only upto B0 but below WA is required for later
- * steppings also so this is extended as well.
- */
- /* WaEnableGuCBootHashCheckNotSet:skl,bxt */
- for (retries = 3; ; ) {
- /*
- * Always reset the GuC just before (re)loading, so
- * that the state and timing are fairly predictable
- */
- err = guc_hw_reset(dev_priv);
- if (err)
- goto fail;
-
- intel_huc_load(dev_priv);
- err = guc_ucode_xfer(dev_priv);
- if (!err)
- break;
-
- if (--retries == 0)
- goto fail;
-
- DRM_INFO("GuC fw load failed: %d; will reset and "
- "retry %d more time(s)\n", err, retries);
- }
-
- guc_fw->load_status = INTEL_UC_FIRMWARE_SUCCESS;
+ if (ret)
+ return -EAGAIN;
- DRM_DEBUG_DRIVER("GuC fw status: fetch %s, load %s\n",
- intel_uc_fw_status_repr(guc_fw->fetch_status),
- intel_uc_fw_status_repr(guc_fw->load_status));
+ guc->fw.load_status = INTEL_UC_FIRMWARE_SUCCESS;
- intel_guc_auth_huc(dev_priv);
-
- if (i915.enable_guc_submission) {
- if (i915.guc_log_level >= 0)
- gen9_enable_guc_interrupts(dev_priv);
-
- err = i915_guc_submission_enable(dev_priv);
- if (err)
- goto fail;
- guc_interrupts_capture(dev_priv);
- }
+ DRM_INFO("GuC %s (firmware %s [version %u.%u])\n",
+ i915.enable_guc_submission ? "submission enabled" : "loaded",
+ guc->fw.path,
+ guc->fw.major_ver_found, guc->fw.minor_ver_found);
return 0;
-
-fail:
- if (guc_fw->load_status == INTEL_UC_FIRMWARE_PENDING)
- guc_fw->load_status = INTEL_UC_FIRMWARE_FAIL;
-
- guc_interrupts_release(dev_priv);
- i915_guc_submission_disable(dev_priv);
- i915_guc_submission_fini(dev_priv);
- i915_ggtt_disable_guc(dev_priv);
-
- /*
- * We've failed to load the firmware :(
- *
- * Decide whether to disable GuC submission and fall back to
- * execlist mode, and whether to hide the error by returning
- * zero or to return -EIO, which the caller will treat as a
- * nonfatal error (i.e. it doesn't prevent driver load, but
- * marks the GPU as wedged until reset).
- */
- if (i915.enable_guc_loading > 1) {
- ret = -EIO;
- } else if (i915.enable_guc_submission > 1) {
- ret = -EIO;
- } else {
- ret = 0;
- }
-
- if (err == 0 && !HAS_GUC_UCODE(dev_priv))
- ; /* Don't mention the GuC! */
- else if (err == 0)
- DRM_INFO("GuC firmware load skipped\n");
- else if (ret != -EIO)
- DRM_NOTE("GuC firmware load failed: %d\n", err);
- else
- DRM_WARN("GuC firmware load failed: %d\n", err);
-
- if (i915.enable_guc_submission) {
- if (fw_path == NULL)
- DRM_INFO("GuC submission without firmware not supported\n");
- if (ret == 0)
- DRM_NOTE("Falling back from GuC submission to execlist mode\n");
- else
- DRM_ERROR("GuC init failed: %d\n", ret);
- }
- i915.enable_guc_submission = 0;
-
- return ret;
-}
-
-void intel_uc_fw_fetch(struct drm_i915_private *dev_priv,
- struct intel_uc_fw *uc_fw)
-{
- struct pci_dev *pdev = dev_priv->drm.pdev;
- struct drm_i915_gem_object *obj;
- const struct firmware *fw = NULL;
- struct uc_css_header *css;
- size_t size;
- int err;
-
- DRM_DEBUG_DRIVER("before requesting firmware: uC fw fetch status %s\n",
- intel_uc_fw_status_repr(uc_fw->fetch_status));
-
- err = request_firmware(&fw, uc_fw->path, &pdev->dev);
- if (err)
- goto fail;
- if (!fw)
- goto fail;
-
- DRM_DEBUG_DRIVER("fetch uC fw from %s succeeded, fw %p\n",
- uc_fw->path, fw);
-
- /* Check the size of the blob before examining buffer contents */
- if (fw->size < sizeof(struct uc_css_header)) {
- DRM_NOTE("Firmware header is missing\n");
- goto fail;
- }
-
- css = (struct uc_css_header *)fw->data;
-
- /* Firmware bits always start from header */
- uc_fw->header_offset = 0;
- uc_fw->header_size = (css->header_size_dw - css->modulus_size_dw -
- css->key_size_dw - css->exponent_size_dw) * sizeof(u32);
-
- if (uc_fw->header_size != sizeof(struct uc_css_header)) {
- DRM_NOTE("CSS header definition mismatch\n");
- goto fail;
- }
-
- /* then, uCode */
- uc_fw->ucode_offset = uc_fw->header_offset + uc_fw->header_size;
- uc_fw->ucode_size = (css->size_dw - css->header_size_dw) * sizeof(u32);
-
- /* now RSA */
- if (css->key_size_dw != UOS_RSA_SCRATCH_MAX_COUNT) {
- DRM_NOTE("RSA key size is bad\n");
- goto fail;
- }
- uc_fw->rsa_offset = uc_fw->ucode_offset + uc_fw->ucode_size;
- uc_fw->rsa_size = css->key_size_dw * sizeof(u32);
-
- /* At least, it should have header, uCode and RSA. Size of all three. */
- size = uc_fw->header_size + uc_fw->ucode_size + uc_fw->rsa_size;
- if (fw->size < size) {
- DRM_NOTE("Missing firmware components\n");
- goto fail;
- }
-
- /*
- * The GuC firmware image has the version number embedded at a well-known
- * offset within the firmware blob; note that major / minor version are
- * TWO bytes each (i.e. u16), although all pointers and offsets are defined
- * in terms of bytes (u8).
- */
- switch (uc_fw->fw) {
- case INTEL_UC_FW_TYPE_GUC:
- /* Header and uCode will be loaded to WOPCM. Size of the two. */
- size = uc_fw->header_size + uc_fw->ucode_size;
-
- /* Top 32k of WOPCM is reserved (8K stack + 24k RC6 context). */
- if (size > intel_guc_wopcm_size(dev_priv)) {
- DRM_ERROR("Firmware is too large to fit in WOPCM\n");
- goto fail;
- }
- uc_fw->major_ver_found = css->guc.sw_version >> 16;
- uc_fw->minor_ver_found = css->guc.sw_version & 0xFFFF;
- break;
-
- case INTEL_UC_FW_TYPE_HUC:
- uc_fw->major_ver_found = css->huc.sw_version >> 16;
- uc_fw->minor_ver_found = css->huc.sw_version & 0xFFFF;
- break;
-
- default:
- DRM_ERROR("Unknown firmware type %d\n", uc_fw->fw);
- err = -ENOEXEC;
- goto fail;
- }
-
- if (uc_fw->major_ver_found != uc_fw->major_ver_wanted ||
- uc_fw->minor_ver_found < uc_fw->minor_ver_wanted) {
- DRM_NOTE("uC firmware version %d.%d, required %d.%d\n",
- uc_fw->major_ver_found, uc_fw->minor_ver_found,
- uc_fw->major_ver_wanted, uc_fw->minor_ver_wanted);
- err = -ENOEXEC;
- goto fail;
- }
-
- DRM_DEBUG_DRIVER("firmware version %d.%d OK (minimum %d.%d)\n",
- uc_fw->major_ver_found, uc_fw->minor_ver_found,
- uc_fw->major_ver_wanted, uc_fw->minor_ver_wanted);
-
- mutex_lock(&dev_priv->drm.struct_mutex);
- obj = i915_gem_object_create_from_data(dev_priv, fw->data, fw->size);
- mutex_unlock(&dev_priv->drm.struct_mutex);
- if (IS_ERR_OR_NULL(obj)) {
- err = obj ? PTR_ERR(obj) : -ENOMEM;
- goto fail;
- }
-
- uc_fw->obj = obj;
- uc_fw->size = fw->size;
-
- DRM_DEBUG_DRIVER("uC fw fetch status SUCCESS, obj %p\n",
- uc_fw->obj);
-
- release_firmware(fw);
- uc_fw->fetch_status = INTEL_UC_FIRMWARE_SUCCESS;
- return;
-
-fail:
- DRM_WARN("Failed to fetch valid uC firmware from %s (error %d)\n",
- uc_fw->path, err);
- DRM_DEBUG_DRIVER("uC fw fetch status FAIL; err %d, fw %p, obj %p\n",
- err, fw, uc_fw->obj);
-
- mutex_lock(&dev_priv->drm.struct_mutex);
- obj = uc_fw->obj;
- if (obj)
- i915_gem_object_put(obj);
- uc_fw->obj = NULL;
- mutex_unlock(&dev_priv->drm.struct_mutex);
-
- release_firmware(fw); /* OK even if fw is NULL */
- uc_fw->fetch_status = INTEL_UC_FIRMWARE_FAIL;
}
/**
- * intel_guc_init() - define parameters and fetch firmware
- * @dev_priv: i915 device private
- *
- * Called early during driver load, but after GEM is initialised.
+ * intel_guc_select_fw() - selects GuC firmware for loading
+ * @guc: intel_guc struct
*
- * The firmware will be transferred to the GuC's memory later,
- * when intel_guc_setup() is called.
+ * Return: zero when we know firmware, non-zero in other case
*/
-void intel_guc_init(struct drm_i915_private *dev_priv)
+int intel_guc_select_fw(struct intel_guc *guc)
{
- struct intel_uc_fw *guc_fw = &dev_priv->guc.fw;
- const char *fw_path;
+ struct drm_i915_private *dev_priv = guc_to_i915(guc);
- if (!HAS_GUC(dev_priv)) {
- i915.enable_guc_loading = 0;
- i915.enable_guc_submission = 0;
- } else {
- /* A negative value means "use platform default" */
- if (i915.enable_guc_loading < 0)
- i915.enable_guc_loading = HAS_GUC_UCODE(dev_priv);
- if (i915.enable_guc_submission < 0)
- i915.enable_guc_submission = HAS_GUC_SCHED(dev_priv);
- }
+ guc->fw.path = NULL;
+ guc->fw.fetch_status = INTEL_UC_FIRMWARE_NONE;
+ guc->fw.load_status = INTEL_UC_FIRMWARE_NONE;
+ guc->fw.type = INTEL_UC_FW_TYPE_GUC;
- if (!HAS_GUC_UCODE(dev_priv)) {
- fw_path = NULL;
+ if (i915.guc_firmware_path) {
+ guc->fw.path = i915.guc_firmware_path;
+ guc->fw.major_ver_wanted = 0;
+ guc->fw.minor_ver_wanted = 0;
} else if (IS_SKYLAKE(dev_priv)) {
- fw_path = I915_SKL_GUC_UCODE;
- guc_fw->major_ver_wanted = SKL_FW_MAJOR;
- guc_fw->minor_ver_wanted = SKL_FW_MINOR;
+ guc->fw.path = I915_SKL_GUC_UCODE;
+ guc->fw.major_ver_wanted = SKL_FW_MAJOR;
+ guc->fw.minor_ver_wanted = SKL_FW_MINOR;
} else if (IS_BROXTON(dev_priv)) {
- fw_path = I915_BXT_GUC_UCODE;
- guc_fw->major_ver_wanted = BXT_FW_MAJOR;
- guc_fw->minor_ver_wanted = BXT_FW_MINOR;
+ guc->fw.path = I915_BXT_GUC_UCODE;
+ guc->fw.major_ver_wanted = BXT_FW_MAJOR;
+ guc->fw.minor_ver_wanted = BXT_FW_MINOR;
} else if (IS_KABYLAKE(dev_priv)) {
- fw_path = I915_KBL_GUC_UCODE;
- guc_fw->major_ver_wanted = KBL_FW_MAJOR;
- guc_fw->minor_ver_wanted = KBL_FW_MINOR;
+ guc->fw.path = I915_KBL_GUC_UCODE;
+ guc->fw.major_ver_wanted = KBL_FW_MAJOR;
+ guc->fw.minor_ver_wanted = KBL_FW_MINOR;
} else {
- fw_path = ""; /* unknown device */
+ DRM_ERROR("No GuC firmware known for platform with GuC!\n");
+ return -ENOENT;
}
- guc_fw->path = fw_path;
- guc_fw->fetch_status = INTEL_UC_FIRMWARE_NONE;
- guc_fw->load_status = INTEL_UC_FIRMWARE_NONE;
-
- /* Early (and silent) return if GuC loading is disabled */
- if (!i915.enable_guc_loading)
- return;
- if (fw_path == NULL)
- return;
- if (*fw_path == '\0')
- return;
-
- guc_fw->fetch_status = INTEL_UC_FIRMWARE_PENDING;
- DRM_DEBUG_DRIVER("GuC firmware pending, path %s\n", fw_path);
- intel_uc_fw_fetch(dev_priv, guc_fw);
- /* status must now be FAIL or SUCCESS */
+ return 0;
}
/**
void intel_guc_fini(struct drm_i915_private *dev_priv)
{
struct intel_uc_fw *guc_fw = &dev_priv->guc.fw;
+ struct drm_i915_gem_object *obj;
mutex_lock(&dev_priv->drm.struct_mutex);
- guc_interrupts_release(dev_priv);
i915_guc_submission_disable(dev_priv);
i915_guc_submission_fini(dev_priv);
-
- if (guc_fw->obj)
- i915_gem_object_put(guc_fw->obj);
- guc_fw->obj = NULL;
mutex_unlock(&dev_priv->drm.struct_mutex);
+ obj = fetch_and_zero(&guc_fw->obj);
+ if (obj)
+ i915_gem_object_put(obj);
+
guc_fw->fetch_status = INTEL_UC_FIRMWARE_NONE;
}
INIT_DELAYED_WORK(&i915->gpu_error.hangcheck_work,
i915_hangcheck_elapsed);
}
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/intel_hangcheck.c"
+#endif
struct drm_device *dev = encoder->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_hdmi *intel_hdmi = enc_to_intel_hdmi(&encoder->base);
- enum intel_display_power_domain power_domain;
u32 tmp;
bool ret;
- power_domain = intel_display_port_power_domain(encoder);
- if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
+ if (!intel_display_power_get_if_enabled(dev_priv,
+ encoder->power_domain))
return false;
ret = false;
ret = true;
out:
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, encoder->power_domain);
return ret;
}
switch (port) {
case PORT_B:
- /*
- * On BXT A0/A1, sw needs to activate DDIA HPD logic and
- * interrupts to check the external panel connection.
- */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1))
- intel_encoder->hpd_pin = HPD_PORT_A;
- else
- intel_encoder->hpd_pin = HPD_PORT_B;
+ intel_encoder->hpd_pin = HPD_PORT_B;
break;
case PORT_C:
intel_encoder->hpd_pin = HPD_PORT_C;
}
intel_encoder->type = INTEL_OUTPUT_HDMI;
+ intel_encoder->power_domain = intel_port_to_power_domain(port);
intel_encoder->port = port;
if (IS_CHERRYVIEW(dev_priv)) {
if (port == PORT_D)
}
#define HPD_STORM_DETECT_PERIOD 1000
-#define HPD_STORM_THRESHOLD 5
#define HPD_STORM_REENABLE_DELAY (2 * 60 * 1000)
/**
* storms. Only the pin specific stats and state are changed, the caller is
* responsible for further action.
*
- * @HPD_STORM_THRESHOLD irqs are allowed within @HPD_STORM_DETECT_PERIOD ms,
- * otherwise it's considered an irq storm, and the irq state is set to
- * @HPD_MARK_DISABLED.
+ * The number of irqs that are allowed within @HPD_STORM_DETECT_PERIOD is
+ * stored in @dev_priv->hotplug.hpd_storm_threshold which defaults to
+ * @HPD_STORM_DEFAULT_THRESHOLD. If this threshold is exceeded, it's
+ * considered an irq storm and the irq state is set to @HPD_MARK_DISABLED.
+ *
+ * The HPD threshold can be controlled through i915_hpd_storm_ctl in debugfs,
+ * and should only be adjusted for automated hotplug testing.
*
* Return true if an irq storm was detected on @pin.
*/
{
unsigned long start = dev_priv->hotplug.stats[pin].last_jiffies;
unsigned long end = start + msecs_to_jiffies(HPD_STORM_DETECT_PERIOD);
+ const int threshold = dev_priv->hotplug.hpd_storm_threshold;
bool storm = false;
if (!time_in_range(jiffies, start, end)) {
dev_priv->hotplug.stats[pin].last_jiffies = jiffies;
dev_priv->hotplug.stats[pin].count = 0;
DRM_DEBUG_KMS("Received HPD interrupt on PIN %d - cnt: 0\n", pin);
- } else if (dev_priv->hotplug.stats[pin].count > HPD_STORM_THRESHOLD) {
+ } else if (dev_priv->hotplug.stats[pin].count > threshold &&
+ threshold) {
dev_priv->hotplug.stats[pin].state = HPD_MARK_DISABLED;
DRM_DEBUG_KMS("HPD interrupt storm detected on PIN %d\n", pin);
storm = true;
static void intel_hpd_irq_storm_disable(struct drm_i915_private *dev_priv)
{
struct drm_device *dev = &dev_priv->drm;
- struct drm_mode_config *mode_config = &dev->mode_config;
struct intel_connector *intel_connector;
struct intel_encoder *intel_encoder;
struct drm_connector *connector;
+ struct drm_connector_list_iter conn_iter;
enum hpd_pin pin;
bool hpd_disabled = false;
- assert_spin_locked(&dev_priv->irq_lock);
+ lockdep_assert_held(&dev_priv->irq_lock);
- list_for_each_entry(connector, &mode_config->connector_list, head) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->polled != DRM_CONNECTOR_POLL_HPD)
continue;
| DRM_CONNECTOR_POLL_DISCONNECT;
hpd_disabled = true;
}
+ drm_connector_list_iter_end(&conn_iter);
/* Enable polling and queue hotplug re-enabling. */
if (hpd_disabled) {
container_of(work, typeof(*dev_priv),
hotplug.reenable_work.work);
struct drm_device *dev = &dev_priv->drm;
- struct drm_mode_config *mode_config = &dev->mode_config;
int i;
intel_runtime_pm_get(dev_priv);
spin_lock_irq(&dev_priv->irq_lock);
for_each_hpd_pin(i) {
struct drm_connector *connector;
+ struct drm_connector_list_iter conn_iter;
if (dev_priv->hotplug.stats[i].state != HPD_DISABLED)
continue;
dev_priv->hotplug.stats[i].state = HPD_ENABLED;
- list_for_each_entry(connector, &mode_config->connector_list, head) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_for_each_connector_iter(connector, &conn_iter) {
struct intel_connector *intel_connector = to_intel_connector(connector);
if (intel_connector->encoder->hpd_pin == i) {
connector->polled = DRM_CONNECTOR_POLL_HPD;
}
}
+ drm_connector_list_iter_end(&conn_iter);
}
if (dev_priv->display_irqs_enabled && dev_priv->display.hpd_irq_setup)
dev_priv->display.hpd_irq_setup(dev_priv);
struct drm_i915_private *dev_priv =
container_of(work, struct drm_i915_private, hotplug.hotplug_work);
struct drm_device *dev = &dev_priv->drm;
- struct drm_mode_config *mode_config = &dev->mode_config;
struct intel_connector *intel_connector;
struct intel_encoder *intel_encoder;
struct drm_connector *connector;
+ struct drm_connector_list_iter conn_iter;
bool changed = false;
u32 hpd_event_bits;
- mutex_lock(&mode_config->mutex);
+ mutex_lock(&dev->mode_config.mutex);
DRM_DEBUG_KMS("running encoder hotplug functions\n");
spin_lock_irq(&dev_priv->irq_lock);
spin_unlock_irq(&dev_priv->irq_lock);
- list_for_each_entry(connector, &mode_config->connector_list, head) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_for_each_connector_iter(connector, &conn_iter) {
intel_connector = to_intel_connector(connector);
if (!intel_connector->encoder)
continue;
changed = true;
}
}
- mutex_unlock(&mode_config->mutex);
+ drm_connector_list_iter_end(&conn_iter);
+ mutex_unlock(&dev->mode_config.mutex);
if (changed)
drm_kms_helper_hotplug_event(dev);
container_of(work, struct drm_i915_private,
hotplug.poll_init_work);
struct drm_device *dev = &dev_priv->drm;
- struct drm_mode_config *mode_config = &dev->mode_config;
struct drm_connector *connector;
+ struct drm_connector_list_iter conn_iter;
bool enabled;
mutex_lock(&dev->mode_config.mutex);
enabled = READ_ONCE(dev_priv->hotplug.poll_enabled);
- list_for_each_entry(connector, &mode_config->connector_list, head) {
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_for_each_connector_iter(connector, &conn_iter) {
struct intel_connector *intel_connector =
to_intel_connector(connector);
connector->polled = intel_connector->polled;
DRM_CONNECTOR_POLL_HPD;
}
}
+ drm_connector_list_iter_end(&conn_iter);
if (enabled)
drm_kms_helper_poll_enable(dev);
}
/**
- * intel_huc_init() - initiate HuC firmware loading request
- * @dev_priv: the drm_i915_private device
- *
- * Called early during driver load, but after GEM is initialised. The loading
- * will continue only when driver explicitly specify firmware name and version.
- * All other cases are considered as INTEL_UC_FIRMWARE_NONE either because HW
- * is not capable or driver yet support it. And there will be no error message
- * for INTEL_UC_FIRMWARE_NONE cases.
- *
- * The DMA-copying to HW is done later when intel_huc_load() is called.
+ * intel_huc_select_fw() - selects HuC firmware for loading
+ * @huc: intel_huc struct
*/
-void intel_huc_init(struct drm_i915_private *dev_priv)
+void intel_huc_select_fw(struct intel_huc *huc)
{
- struct intel_huc *huc = &dev_priv->huc;
- struct intel_uc_fw *huc_fw = &huc->fw;
- const char *fw_path = NULL;
-
- huc_fw->path = NULL;
- huc_fw->fetch_status = INTEL_UC_FIRMWARE_NONE;
- huc_fw->load_status = INTEL_UC_FIRMWARE_NONE;
- huc_fw->fw = INTEL_UC_FW_TYPE_HUC;
-
- if (!HAS_HUC_UCODE(dev_priv))
- return;
-
- if (IS_SKYLAKE(dev_priv)) {
- fw_path = I915_SKL_HUC_UCODE;
- huc_fw->major_ver_wanted = SKL_HUC_FW_MAJOR;
- huc_fw->minor_ver_wanted = SKL_HUC_FW_MINOR;
+ struct drm_i915_private *dev_priv = huc_to_i915(huc);
+
+ huc->fw.path = NULL;
+ huc->fw.fetch_status = INTEL_UC_FIRMWARE_NONE;
+ huc->fw.load_status = INTEL_UC_FIRMWARE_NONE;
+ huc->fw.type = INTEL_UC_FW_TYPE_HUC;
+
+ if (i915.huc_firmware_path) {
+ huc->fw.path = i915.huc_firmware_path;
+ huc->fw.major_ver_wanted = 0;
+ huc->fw.minor_ver_wanted = 0;
+ } else if (IS_SKYLAKE(dev_priv)) {
+ huc->fw.path = I915_SKL_HUC_UCODE;
+ huc->fw.major_ver_wanted = SKL_HUC_FW_MAJOR;
+ huc->fw.minor_ver_wanted = SKL_HUC_FW_MINOR;
} else if (IS_BROXTON(dev_priv)) {
- fw_path = I915_BXT_HUC_UCODE;
- huc_fw->major_ver_wanted = BXT_HUC_FW_MAJOR;
- huc_fw->minor_ver_wanted = BXT_HUC_FW_MINOR;
+ huc->fw.path = I915_BXT_HUC_UCODE;
+ huc->fw.major_ver_wanted = BXT_HUC_FW_MAJOR;
+ huc->fw.minor_ver_wanted = BXT_HUC_FW_MINOR;
} else if (IS_KABYLAKE(dev_priv)) {
- fw_path = I915_KBL_HUC_UCODE;
- huc_fw->major_ver_wanted = KBL_HUC_FW_MAJOR;
- huc_fw->minor_ver_wanted = KBL_HUC_FW_MINOR;
+ huc->fw.path = I915_KBL_HUC_UCODE;
+ huc->fw.major_ver_wanted = KBL_HUC_FW_MAJOR;
+ huc->fw.minor_ver_wanted = KBL_HUC_FW_MINOR;
+ } else {
+ DRM_ERROR("No HuC firmware known for platform with HuC!\n");
+ return;
}
-
- huc_fw->path = fw_path;
- huc_fw->fetch_status = INTEL_UC_FIRMWARE_PENDING;
-
- DRM_DEBUG_DRIVER("HuC firmware pending, path %s\n", fw_path);
-
- WARN(huc_fw->path == NULL, "HuC present but no fw path\n");
-
- intel_uc_fw_fetch(dev_priv, huc_fw);
}
/**
- * intel_huc_load() - load HuC uCode to device
- * @dev_priv: the drm_i915_private device
+ * intel_huc_init_hw() - load HuC uCode to device
+ * @huc: intel_huc structure
*
* Called from guc_setup() during driver loading and also after a GPU reset.
* Be note that HuC loading must be done before GuC loading.
*
* Return: non-zero code on error
*/
-int intel_huc_load(struct drm_i915_private *dev_priv)
+int intel_huc_init_hw(struct intel_huc *huc)
{
- struct intel_uc_fw *huc_fw = &dev_priv->huc.fw;
+ struct drm_i915_private *dev_priv = huc_to_i915(huc);
int err;
- if (huc_fw->fetch_status == INTEL_UC_FIRMWARE_NONE)
+ if (huc->fw.fetch_status == INTEL_UC_FIRMWARE_NONE)
return 0;
DRM_DEBUG_DRIVER("%s fw status: fetch %s, load %s\n",
- huc_fw->path,
- intel_uc_fw_status_repr(huc_fw->fetch_status),
- intel_uc_fw_status_repr(huc_fw->load_status));
+ huc->fw.path,
+ intel_uc_fw_status_repr(huc->fw.fetch_status),
+ intel_uc_fw_status_repr(huc->fw.load_status));
- if (huc_fw->fetch_status == INTEL_UC_FIRMWARE_SUCCESS &&
- huc_fw->load_status == INTEL_UC_FIRMWARE_FAIL)
+ if (huc->fw.fetch_status == INTEL_UC_FIRMWARE_SUCCESS &&
+ huc->fw.load_status == INTEL_UC_FIRMWARE_FAIL)
return -ENOEXEC;
- huc_fw->load_status = INTEL_UC_FIRMWARE_PENDING;
+ huc->fw.load_status = INTEL_UC_FIRMWARE_PENDING;
- switch (huc_fw->fetch_status) {
+ switch (huc->fw.fetch_status) {
case INTEL_UC_FIRMWARE_FAIL:
/* something went wrong :( */
err = -EIO;
default:
/* "can't happen" */
WARN_ONCE(1, "HuC fw %s invalid fetch_status %s [%d]\n",
- huc_fw->path,
- intel_uc_fw_status_repr(huc_fw->fetch_status),
- huc_fw->fetch_status);
+ huc->fw.path,
+ intel_uc_fw_status_repr(huc->fw.fetch_status),
+ huc->fw.fetch_status);
err = -ENXIO;
goto fail;
if (err)
goto fail;
- huc_fw->load_status = INTEL_UC_FIRMWARE_SUCCESS;
+ huc->fw.load_status = INTEL_UC_FIRMWARE_SUCCESS;
DRM_DEBUG_DRIVER("%s fw status: fetch %s, load %s\n",
- huc_fw->path,
- intel_uc_fw_status_repr(huc_fw->fetch_status),
- intel_uc_fw_status_repr(huc_fw->load_status));
+ huc->fw.path,
+ intel_uc_fw_status_repr(huc->fw.fetch_status),
+ intel_uc_fw_status_repr(huc->fw.load_status));
return 0;
fail:
- if (huc_fw->load_status == INTEL_UC_FIRMWARE_PENDING)
- huc_fw->load_status = INTEL_UC_FIRMWARE_FAIL;
+ if (huc->fw.load_status == INTEL_UC_FIRMWARE_PENDING)
+ huc->fw.load_status = INTEL_UC_FIRMWARE_FAIL;
DRM_ERROR("Failed to complete HuC uCode load with ret %d\n", err);
void intel_huc_fini(struct drm_i915_private *dev_priv)
{
struct intel_uc_fw *huc_fw = &dev_priv->huc.fw;
+ struct drm_i915_gem_object *obj;
- mutex_lock(&dev_priv->drm.struct_mutex);
- if (huc_fw->obj)
- i915_gem_object_put(huc_fw->obj);
- huc_fw->obj = NULL;
- mutex_unlock(&dev_priv->drm.struct_mutex);
+ obj = fetch_and_zero(&huc_fw->obj);
+ if (obj)
+ i915_gem_object_put(obj);
huc_fw->fetch_status = INTEL_UC_FIRMWARE_NONE;
}
{
if (IS_GEN9_LP(dev_priv))
return &gmbus_pins_bxt[pin];
- else if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ else if (IS_GEN9_BC(dev_priv))
return &gmbus_pins_skl[pin];
else if (IS_BROADWELL(dev_priv))
return &gmbus_pins_bdw[pin];
if (IS_GEN9_LP(dev_priv))
size = ARRAY_SIZE(gmbus_pins_bxt);
- else if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ else if (IS_GEN9_BC(dev_priv))
size = ARRAY_SIZE(gmbus_pins_skl);
else if (IS_BROADWELL(dev_priv))
size = ARRAY_SIZE(gmbus_pins_bdw);
#define CTX_R_PWR_CLK_STATE 0x42
#define CTX_GPGPU_CSR_BASE_ADDRESS 0x44
-#define GEN8_CTX_VALID (1<<0)
-#define GEN8_CTX_FORCE_PD_RESTORE (1<<1)
-#define GEN8_CTX_FORCE_RESTORE (1<<2)
-#define GEN8_CTX_L3LLC_COHERENT (1<<5)
-#define GEN8_CTX_PRIVILEGE (1<<8)
-
-#define ASSIGN_CTX_REG(reg_state, pos, reg, val) do { \
+#define CTX_REG(reg_state, pos, reg, val) do { \
(reg_state)[(pos)+0] = i915_mmio_reg_offset(reg); \
(reg_state)[(pos)+1] = (val); \
} while (0)
reg_state[CTX_PDP0_LDW + 1] = lower_32_bits(px_dma(&ppgtt->pml4)); \
} while (0)
-enum {
- FAULT_AND_HANG = 0,
- FAULT_AND_HALT, /* Debug only */
- FAULT_AND_STREAM,
- FAULT_AND_CONTINUE /* Unsupported */
-};
-#define GEN8_CTX_ID_SHIFT 32
-#define GEN8_CTX_ID_WIDTH 21
#define GEN8_CTX_RCS_INDIRECT_CTX_OFFSET_DEFAULT 0x17
#define GEN9_CTX_RCS_INDIRECT_CTX_OFFSET_DEFAULT 0x26
return 0;
}
-static void
-logical_ring_init_platform_invariants(struct intel_engine_cs *engine)
-{
- struct drm_i915_private *dev_priv = engine->i915;
-
- engine->disable_lite_restore_wa =
- IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1) &&
- (engine->id == VCS || engine->id == VCS2);
-
- engine->ctx_desc_template = GEN8_CTX_VALID;
- if (IS_GEN8(dev_priv))
- engine->ctx_desc_template |= GEN8_CTX_L3LLC_COHERENT;
- engine->ctx_desc_template |= GEN8_CTX_PRIVILEGE;
-
- /* TODO: WaDisableLiteRestore when we start using semaphore
- * signalling between Command Streamers */
- /* ring->ctx_desc_template |= GEN8_CTX_FORCE_RESTORE; */
-
- /* WaEnableForceRestoreInCtxtDescForVCS:skl */
- /* WaEnableForceRestoreInCtxtDescForVCS:bxt */
- if (engine->disable_lite_restore_wa)
- engine->ctx_desc_template |= GEN8_CTX_FORCE_RESTORE;
-}
-
/**
* intel_lr_context_descriptor_update() - calculate & cache the descriptor
* descriptor for a pinned context
*
* This is what a descriptor looks like, from LSB to MSB::
*
- * bits 0-11: flags, GEN8_CTX_* (cached in ctx_desc_template)
+ * bits 0-11: flags, GEN8_CTX_* (cached in ctx->desc_template)
* bits 12-31: LRCA, GTT address of (the HWSP of) this context
* bits 32-52: ctx ID, a globally unique tag
* bits 53-54: mbz, reserved for use by hardware
BUILD_BUG_ON(MAX_CONTEXT_HW_ID > (1<<GEN8_CTX_ID_WIDTH));
- desc = ctx->desc_template; /* bits 3-4 */
- desc |= engine->ctx_desc_template; /* bits 0-11 */
+ desc = ctx->desc_template; /* bits 0-11 */
desc |= i915_ggtt_offset(ce->state) + LRC_PPHWSP_PN * PAGE_SIZE;
/* bits 12-31 */
desc |= (u64)ctx->hw_id << GEN8_CTX_ID_SHIFT; /* bits 32-52 */
rq->ctx->ppgtt ?: rq->i915->mm.aliasing_ppgtt;
u32 *reg_state = ce->lrc_reg_state;
+ GEM_BUG_ON(!IS_ALIGNED(rq->tail, 8));
reg_state[CTX_RING_TAIL+1] = rq->tail;
/* True 32b PPGTT with dynamic page allocation: update PDP
* PML4 is allocated during ppgtt init, so this is not needed
* in 48-bit mode.
*/
- if (ppgtt && !USES_FULL_48BIT_PPGTT(ppgtt->base.dev))
+ if (ppgtt && !i915_vm_is_48bit(&ppgtt->base))
execlists_update_context_pdps(ppgtt, reg_state);
return ce->lrc_desc;
dev_priv->regs + i915_mmio_reg_offset(RING_ELSP(engine));
u64 desc[2];
+ GEM_BUG_ON(port[0].count > 1);
if (!port[0].count)
execlists_context_status_change(port[0].request,
INTEL_CONTEXT_SCHEDULE_IN);
desc[0] = execlists_update_context(port[0].request);
- engine->preempt_wa = port[0].count++; /* bdw only? fixed on skl? */
+ GEM_DEBUG_EXEC(port[0].context_id = upper_32_bits(desc[0]));
+ port[0].count++;
if (port[1].request) {
GEM_BUG_ON(port[1].count);
execlists_context_status_change(port[1].request,
INTEL_CONTEXT_SCHEDULE_IN);
desc[1] = execlists_update_context(port[1].request);
+ GEM_DEBUG_EXEC(port[1].context_id = upper_32_bits(desc[1]));
port[1].count = 1;
} else {
desc[1] = 0;
struct rb_node *rb;
bool submit = false;
+ /* After execlist_first is updated, the tasklet will be rescheduled.
+ *
+ * If we are currently running (inside the tasklet) and a third
+ * party queues a request and so updates engine->execlist_first under
+ * the spinlock (which we have elided), it will atomically set the
+ * TASKLET_SCHED flag causing the us to be re-executed and pick up
+ * the change in state (the update to TASKLET_SCHED incurs a memory
+ * barrier making this cross-cpu checking safe).
+ */
+ if (!READ_ONCE(engine->execlist_first))
+ return;
+
last = port->request;
if (last)
/* WaIdleLiteRestore:bdw,skl
cursor->priotree.priority = INT_MAX;
__i915_gem_request_submit(cursor);
+ trace_i915_gem_request_in(cursor, port - engine->execlist_port);
last = cursor;
submit = true;
}
return !engine->execlist_port[0].request;
}
-/**
- * intel_execlists_idle() - Determine if all engine submission ports are idle
- * @dev_priv: i915 device private
- *
- * Return true if there are no requests pending on any of the submission ports
- * of any engines.
- */
-bool intel_execlists_idle(struct drm_i915_private *dev_priv)
+static bool execlists_elsp_ready(const struct intel_engine_cs *engine)
{
- struct intel_engine_cs *engine;
- enum intel_engine_id id;
-
- if (!i915.enable_execlists)
- return true;
-
- for_each_engine(engine, dev_priv, id)
- if (!execlists_elsp_idle(engine))
- return false;
-
- return true;
-}
-
-static bool execlists_elsp_ready(struct intel_engine_cs *engine)
-{
- int port;
-
- port = 1; /* wait for a free slot */
- if (engine->disable_lite_restore_wa || engine->preempt_wa)
- port = 0; /* wait for GPU to be idle before continuing */
+ const struct execlist_port *port = engine->execlist_port;
- return !engine->execlist_port[port].request;
+ return port[0].count + port[1].count < 2;
}
/*
intel_uncore_forcewake_get(dev_priv, engine->fw_domains);
- if (!execlists_elsp_idle(engine)) {
+ while (test_and_clear_bit(ENGINE_IRQ_EXECLIST, &engine->irq_posted)) {
u32 __iomem *csb_mmio =
dev_priv->regs + i915_mmio_reg_offset(RING_CONTEXT_STATUS_PTR(engine));
u32 __iomem *buf =
csb = readl(csb_mmio);
head = GEN8_CSB_READ_PTR(csb);
tail = GEN8_CSB_WRITE_PTR(csb);
+ if (head == tail)
+ break;
+
if (tail < head)
tail += GEN8_CSB_ENTRIES;
- while (head < tail) {
+ do {
unsigned int idx = ++head % GEN8_CSB_ENTRIES;
unsigned int status = readl(buf + 2 * idx);
+ /* We are flying near dragons again.
+ *
+ * We hold a reference to the request in execlist_port[]
+ * but no more than that. We are operating in softirq
+ * context and so cannot hold any mutex or sleep. That
+ * prevents us stopping the requests we are processing
+ * in port[] from being retired simultaneously (the
+ * breadcrumb will be complete before we see the
+ * context-switch). As we only hold the reference to the
+ * request, any pointer chasing underneath the request
+ * is subject to a potential use-after-free. Thus we
+ * store all of the bookkeeping within port[] as
+ * required, and avoid using unguarded pointers beneath
+ * request itself. The same applies to the atomic
+ * status notifier.
+ */
+
if (!(status & GEN8_CTX_STATUS_COMPLETED_MASK))
continue;
+ /* Check the context/desc id for this event matches */
+ GEM_DEBUG_BUG_ON(readl(buf + 2 * idx + 1) !=
+ port[0].context_id);
+
GEM_BUG_ON(port[0].count == 0);
if (--port[0].count == 0) {
GEM_BUG_ON(status & GEN8_CTX_STATUS_PREEMPTED);
+ GEM_BUG_ON(!i915_gem_request_completed(port[0].request));
execlists_context_status_change(port[0].request,
INTEL_CONTEXT_SCHEDULE_OUT);
+ trace_i915_gem_request_out(port[0].request);
i915_gem_request_put(port[0].request);
port[0] = port[1];
memset(&port[1], 0, sizeof(port[1]));
-
- engine->preempt_wa = false;
}
GEM_BUG_ON(port[0].count == 0 &&
!(status & GEN8_CTX_STATUS_ACTIVE_IDLE));
- }
+ } while (head < tail);
writel(_MASKED_FIELD(GEN8_CSB_READ_PTR_MASK,
GEN8_CSB_WRITE_PTR(csb) << 8),
/* Will be called from irq-context when using foreign fences. */
spin_lock_irqsave(&engine->timeline->lock, flags);
- if (insert_request(&request->priotree, &engine->execlist_queue))
+ if (insert_request(&request->priotree, &engine->execlist_queue)) {
engine->execlist_first = &request->priotree.node;
- if (execlists_elsp_idle(engine))
- tasklet_hi_schedule(&engine->irq_tasklet);
+ if (execlists_elsp_ready(engine))
+ tasklet_hi_schedule(&engine->irq_tasklet);
+ }
spin_unlock_irqrestore(&engine->timeline->lock, flags);
}
if (ce->pin_count++)
return 0;
+ GEM_BUG_ON(!ce->pin_count); /* no overflow please! */
if (!ce->state) {
ret = execlists_context_deferred_alloc(ctx, engine);
}
GEM_BUG_ON(!ce->state);
- flags = PIN_GLOBAL;
+ flags = PIN_GLOBAL | PIN_HIGH;
if (ctx->ggtt_offset_bias)
flags |= PIN_OFFSET_BIAS | ctx->ggtt_offset_bias;
- if (i915_gem_context_is_kernel(ctx))
- flags |= PIN_HIGH;
ret = i915_vma_pin(ce->state, 0, GEN8_LR_CONTEXT_ALIGN, flags);
if (ret)
{
struct intel_engine_cs *engine = request->engine;
struct intel_context *ce = &request->ctx->engine[engine->id];
+ u32 *cs;
int ret;
GEM_BUG_ON(!ce->pin_count);
goto err;
}
- ret = intel_ring_begin(request, 0);
- if (ret)
+ cs = intel_ring_begin(request, 0);
+ if (IS_ERR(cs)) {
+ ret = PTR_ERR(cs);
goto err_unreserve;
+ }
if (!ce->initialised) {
ret = engine->init_context(request);
return ret;
}
-static int intel_logical_ring_workarounds_emit(struct drm_i915_gem_request *req)
-{
- int ret, i;
- struct intel_ring *ring = req->ring;
- struct i915_workarounds *w = &req->i915->workarounds;
-
- if (w->count == 0)
- return 0;
-
- ret = req->engine->emit_flush(req, EMIT_BARRIER);
- if (ret)
- return ret;
-
- ret = intel_ring_begin(req, w->count * 2 + 2);
- if (ret)
- return ret;
-
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(w->count));
- for (i = 0; i < w->count; i++) {
- intel_ring_emit_reg(ring, w->reg[i].addr);
- intel_ring_emit(ring, w->reg[i].value);
- }
- intel_ring_emit(ring, MI_NOOP);
-
- intel_ring_advance(ring);
-
- ret = req->engine->emit_flush(req, EMIT_BARRIER);
- if (ret)
- return ret;
-
- return 0;
-}
-
-#define wa_ctx_emit(batch, index, cmd) \
- do { \
- int __index = (index)++; \
- if (WARN_ON(__index >= (PAGE_SIZE / sizeof(uint32_t)))) { \
- return -ENOSPC; \
- } \
- batch[__index] = (cmd); \
- } while (0)
-
-#define wa_ctx_emit_reg(batch, index, reg) \
- wa_ctx_emit((batch), (index), i915_mmio_reg_offset(reg))
-
/*
* In this WA we need to set GEN8_L3SQCREG4[21:21] and reset it after
* PIPE_CONTROL instruction. This is required for the flush to happen correctly
* This WA is also required for Gen9 so extracting as a function avoids
* code duplication.
*/
-static inline int gen8_emit_flush_coherentl3_wa(struct intel_engine_cs *engine,
- uint32_t *batch,
- uint32_t index)
+static u32 *
+gen8_emit_flush_coherentl3_wa(struct intel_engine_cs *engine, u32 *batch)
{
- uint32_t l3sqc4_flush = (0x40400000 | GEN8_LQSC_FLUSH_COHERENT_LINES);
-
- wa_ctx_emit(batch, index, (MI_STORE_REGISTER_MEM_GEN8 |
- MI_SRM_LRM_GLOBAL_GTT));
- wa_ctx_emit_reg(batch, index, GEN8_L3SQCREG4);
- wa_ctx_emit(batch, index, i915_ggtt_offset(engine->scratch) + 256);
- wa_ctx_emit(batch, index, 0);
-
- wa_ctx_emit(batch, index, MI_LOAD_REGISTER_IMM(1));
- wa_ctx_emit_reg(batch, index, GEN8_L3SQCREG4);
- wa_ctx_emit(batch, index, l3sqc4_flush);
-
- wa_ctx_emit(batch, index, GFX_OP_PIPE_CONTROL(6));
- wa_ctx_emit(batch, index, (PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_DC_FLUSH_ENABLE));
- wa_ctx_emit(batch, index, 0);
- wa_ctx_emit(batch, index, 0);
- wa_ctx_emit(batch, index, 0);
- wa_ctx_emit(batch, index, 0);
-
- wa_ctx_emit(batch, index, (MI_LOAD_REGISTER_MEM_GEN8 |
- MI_SRM_LRM_GLOBAL_GTT));
- wa_ctx_emit_reg(batch, index, GEN8_L3SQCREG4);
- wa_ctx_emit(batch, index, i915_ggtt_offset(engine->scratch) + 256);
- wa_ctx_emit(batch, index, 0);
-
- return index;
-}
+ *batch++ = MI_STORE_REGISTER_MEM_GEN8 | MI_SRM_LRM_GLOBAL_GTT;
+ *batch++ = i915_mmio_reg_offset(GEN8_L3SQCREG4);
+ *batch++ = i915_ggtt_offset(engine->scratch) + 256;
+ *batch++ = 0;
-static inline uint32_t wa_ctx_start(struct i915_wa_ctx_bb *wa_ctx,
- uint32_t offset,
- uint32_t start_alignment)
-{
- return wa_ctx->offset = ALIGN(offset, start_alignment);
-}
+ *batch++ = MI_LOAD_REGISTER_IMM(1);
+ *batch++ = i915_mmio_reg_offset(GEN8_L3SQCREG4);
+ *batch++ = 0x40400000 | GEN8_LQSC_FLUSH_COHERENT_LINES;
-static inline int wa_ctx_end(struct i915_wa_ctx_bb *wa_ctx,
- uint32_t offset,
- uint32_t size_alignment)
-{
- wa_ctx->size = offset - wa_ctx->offset;
+ batch = gen8_emit_pipe_control(batch,
+ PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_DC_FLUSH_ENABLE,
+ 0);
- WARN(wa_ctx->size % size_alignment,
- "wa_ctx_bb failed sanity checks: size %d is not aligned to %d\n",
- wa_ctx->size, size_alignment);
- return 0;
+ *batch++ = MI_LOAD_REGISTER_MEM_GEN8 | MI_SRM_LRM_GLOBAL_GTT;
+ *batch++ = i915_mmio_reg_offset(GEN8_L3SQCREG4);
+ *batch++ = i915_ggtt_offset(engine->scratch) + 256;
+ *batch++ = 0;
+
+ return batch;
}
/*
* MI_BATCH_BUFFER_END will be added to perctx batch and both of them together
* makes a complete batch buffer.
*/
-static int gen8_init_indirectctx_bb(struct intel_engine_cs *engine,
- struct i915_wa_ctx_bb *wa_ctx,
- uint32_t *batch,
- uint32_t *offset)
+static u32 *gen8_init_indirectctx_bb(struct intel_engine_cs *engine, u32 *batch)
{
- uint32_t scratch_addr;
- uint32_t index = wa_ctx_start(wa_ctx, *offset, CACHELINE_DWORDS);
-
/* WaDisableCtxRestoreArbitration:bdw,chv */
- wa_ctx_emit(batch, index, MI_ARB_ON_OFF | MI_ARB_DISABLE);
+ *batch++ = MI_ARB_ON_OFF | MI_ARB_DISABLE;
/* WaFlushCoherentL3CacheLinesAtContextSwitch:bdw */
- if (IS_BROADWELL(engine->i915)) {
- int rc = gen8_emit_flush_coherentl3_wa(engine, batch, index);
- if (rc < 0)
- return rc;
- index = rc;
- }
+ if (IS_BROADWELL(engine->i915))
+ batch = gen8_emit_flush_coherentl3_wa(engine, batch);
/* WaClearSlmSpaceAtContextSwitch:bdw,chv */
/* Actual scratch location is at 128 bytes offset */
- scratch_addr = i915_ggtt_offset(engine->scratch) + 2 * CACHELINE_BYTES;
-
- wa_ctx_emit(batch, index, GFX_OP_PIPE_CONTROL(6));
- wa_ctx_emit(batch, index, (PIPE_CONTROL_FLUSH_L3 |
- PIPE_CONTROL_GLOBAL_GTT_IVB |
- PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_QW_WRITE));
- wa_ctx_emit(batch, index, scratch_addr);
- wa_ctx_emit(batch, index, 0);
- wa_ctx_emit(batch, index, 0);
- wa_ctx_emit(batch, index, 0);
+ batch = gen8_emit_pipe_control(batch,
+ PIPE_CONTROL_FLUSH_L3 |
+ PIPE_CONTROL_GLOBAL_GTT_IVB |
+ PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_QW_WRITE,
+ i915_ggtt_offset(engine->scratch) +
+ 2 * CACHELINE_BYTES);
/* Pad to end of cacheline */
- while (index % CACHELINE_DWORDS)
- wa_ctx_emit(batch, index, MI_NOOP);
+ while ((unsigned long)batch % CACHELINE_BYTES)
+ *batch++ = MI_NOOP;
/*
* MI_BATCH_BUFFER_END is not required in Indirect ctx BB because
* in the register CTX_RCS_INDIRECT_CTX
*/
- return wa_ctx_end(wa_ctx, *offset = index, CACHELINE_DWORDS);
+ return batch;
}
/*
* This batch is terminated with MI_BATCH_BUFFER_END and so we need not add padding
* to align it with cacheline as padding after MI_BATCH_BUFFER_END is redundant.
*/
-static int gen8_init_perctx_bb(struct intel_engine_cs *engine,
- struct i915_wa_ctx_bb *wa_ctx,
- uint32_t *batch,
- uint32_t *offset)
+static u32 *gen8_init_perctx_bb(struct intel_engine_cs *engine, u32 *batch)
{
- uint32_t index = wa_ctx_start(wa_ctx, *offset, CACHELINE_DWORDS);
-
/* WaDisableCtxRestoreArbitration:bdw,chv */
- wa_ctx_emit(batch, index, MI_ARB_ON_OFF | MI_ARB_ENABLE);
-
- wa_ctx_emit(batch, index, MI_BATCH_BUFFER_END);
+ *batch++ = MI_ARB_ON_OFF | MI_ARB_ENABLE;
+ *batch++ = MI_BATCH_BUFFER_END;
- return wa_ctx_end(wa_ctx, *offset = index, 1);
+ return batch;
}
-static int gen9_init_indirectctx_bb(struct intel_engine_cs *engine,
- struct i915_wa_ctx_bb *wa_ctx,
- uint32_t *batch,
- uint32_t *offset)
+static u32 *gen9_init_indirectctx_bb(struct intel_engine_cs *engine, u32 *batch)
{
- int ret;
- struct drm_i915_private *dev_priv = engine->i915;
- uint32_t index = wa_ctx_start(wa_ctx, *offset, CACHELINE_DWORDS);
+ /* WaFlushCoherentL3CacheLinesAtContextSwitch:skl,bxt,glk */
+ batch = gen8_emit_flush_coherentl3_wa(engine, batch);
- /* WaDisableCtxRestoreArbitration:bxt */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1))
- wa_ctx_emit(batch, index, MI_ARB_ON_OFF | MI_ARB_DISABLE);
-
- /* WaFlushCoherentL3CacheLinesAtContextSwitch:skl,bxt */
- ret = gen8_emit_flush_coherentl3_wa(engine, batch, index);
- if (ret < 0)
- return ret;
- index = ret;
-
- /* WaDisableGatherAtSetShaderCommonSlice:skl,bxt,kbl */
- wa_ctx_emit(batch, index, MI_LOAD_REGISTER_IMM(1));
- wa_ctx_emit_reg(batch, index, COMMON_SLICE_CHICKEN2);
- wa_ctx_emit(batch, index, _MASKED_BIT_DISABLE(
- GEN9_DISABLE_GATHER_AT_SET_SHADER_COMMON_SLICE));
- wa_ctx_emit(batch, index, MI_NOOP);
+ /* WaDisableGatherAtSetShaderCommonSlice:skl,bxt,kbl,glk */
+ *batch++ = MI_LOAD_REGISTER_IMM(1);
+ *batch++ = i915_mmio_reg_offset(COMMON_SLICE_CHICKEN2);
+ *batch++ = _MASKED_BIT_DISABLE(
+ GEN9_DISABLE_GATHER_AT_SET_SHADER_COMMON_SLICE);
+ *batch++ = MI_NOOP;
/* WaClearSlmSpaceAtContextSwitch:kbl */
/* Actual scratch location is at 128 bytes offset */
- if (IS_KBL_REVID(dev_priv, 0, KBL_REVID_A0)) {
- u32 scratch_addr =
- i915_ggtt_offset(engine->scratch) + 2 * CACHELINE_BYTES;
-
- wa_ctx_emit(batch, index, GFX_OP_PIPE_CONTROL(6));
- wa_ctx_emit(batch, index, (PIPE_CONTROL_FLUSH_L3 |
- PIPE_CONTROL_GLOBAL_GTT_IVB |
- PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_QW_WRITE));
- wa_ctx_emit(batch, index, scratch_addr);
- wa_ctx_emit(batch, index, 0);
- wa_ctx_emit(batch, index, 0);
- wa_ctx_emit(batch, index, 0);
+ if (IS_KBL_REVID(engine->i915, 0, KBL_REVID_A0)) {
+ batch = gen8_emit_pipe_control(batch,
+ PIPE_CONTROL_FLUSH_L3 |
+ PIPE_CONTROL_GLOBAL_GTT_IVB |
+ PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_QW_WRITE,
+ i915_ggtt_offset(engine->scratch)
+ + 2 * CACHELINE_BYTES);
}
- /* WaMediaPoolStateCmdInWABB:bxt */
+ /* WaMediaPoolStateCmdInWABB:bxt,glk */
if (HAS_POOLED_EU(engine->i915)) {
/*
* EU pool configuration is setup along with golden context
* possible configurations, to avoid duplication they are
* not shown here again.
*/
- u32 eu_pool_config = 0x00777000;
- wa_ctx_emit(batch, index, GEN9_MEDIA_POOL_STATE);
- wa_ctx_emit(batch, index, GEN9_MEDIA_POOL_ENABLE);
- wa_ctx_emit(batch, index, eu_pool_config);
- wa_ctx_emit(batch, index, 0);
- wa_ctx_emit(batch, index, 0);
- wa_ctx_emit(batch, index, 0);
+ *batch++ = GEN9_MEDIA_POOL_STATE;
+ *batch++ = GEN9_MEDIA_POOL_ENABLE;
+ *batch++ = 0x00777000;
+ *batch++ = 0;
+ *batch++ = 0;
+ *batch++ = 0;
}
/* Pad to end of cacheline */
- while (index % CACHELINE_DWORDS)
- wa_ctx_emit(batch, index, MI_NOOP);
+ while ((unsigned long)batch % CACHELINE_BYTES)
+ *batch++ = MI_NOOP;
- return wa_ctx_end(wa_ctx, *offset = index, CACHELINE_DWORDS);
+ return batch;
}
-static int gen9_init_perctx_bb(struct intel_engine_cs *engine,
- struct i915_wa_ctx_bb *wa_ctx,
- uint32_t *batch,
- uint32_t *offset)
+static u32 *gen9_init_perctx_bb(struct intel_engine_cs *engine, u32 *batch)
{
- uint32_t index = wa_ctx_start(wa_ctx, *offset, CACHELINE_DWORDS);
-
- /* WaSetDisablePixMaskCammingAndRhwoInCommonSliceChicken:bxt */
- if (IS_BXT_REVID(engine->i915, 0, BXT_REVID_A1)) {
- wa_ctx_emit(batch, index, MI_LOAD_REGISTER_IMM(1));
- wa_ctx_emit_reg(batch, index, GEN9_SLICE_COMMON_ECO_CHICKEN0);
- wa_ctx_emit(batch, index,
- _MASKED_BIT_ENABLE(DISABLE_PIXEL_MASK_CAMMING));
- wa_ctx_emit(batch, index, MI_NOOP);
- }
+ *batch++ = MI_BATCH_BUFFER_END;
- /* WaClearTdlStateAckDirtyBits:bxt */
- if (IS_BXT_REVID(engine->i915, 0, BXT_REVID_B0)) {
- wa_ctx_emit(batch, index, MI_LOAD_REGISTER_IMM(4));
-
- wa_ctx_emit_reg(batch, index, GEN8_STATE_ACK);
- wa_ctx_emit(batch, index, _MASKED_BIT_DISABLE(GEN9_SUBSLICE_TDL_ACK_BITS));
-
- wa_ctx_emit_reg(batch, index, GEN9_STATE_ACK_SLICE1);
- wa_ctx_emit(batch, index, _MASKED_BIT_DISABLE(GEN9_SUBSLICE_TDL_ACK_BITS));
-
- wa_ctx_emit_reg(batch, index, GEN9_STATE_ACK_SLICE2);
- wa_ctx_emit(batch, index, _MASKED_BIT_DISABLE(GEN9_SUBSLICE_TDL_ACK_BITS));
-
- wa_ctx_emit_reg(batch, index, GEN7_ROW_CHICKEN2);
- /* dummy write to CS, mask bits are 0 to ensure the register is not modified */
- wa_ctx_emit(batch, index, 0x0);
- wa_ctx_emit(batch, index, MI_NOOP);
- }
-
- /* WaDisableCtxRestoreArbitration:bxt */
- if (IS_BXT_REVID(engine->i915, 0, BXT_REVID_A1))
- wa_ctx_emit(batch, index, MI_ARB_ON_OFF | MI_ARB_ENABLE);
-
- wa_ctx_emit(batch, index, MI_BATCH_BUFFER_END);
-
- return wa_ctx_end(wa_ctx, *offset = index, 1);
+ return batch;
}
-static int lrc_setup_wa_ctx_obj(struct intel_engine_cs *engine, u32 size)
+#define CTX_WA_BB_OBJ_SIZE (PAGE_SIZE)
+
+static int lrc_setup_wa_ctx(struct intel_engine_cs *engine)
{
struct drm_i915_gem_object *obj;
struct i915_vma *vma;
int err;
- obj = i915_gem_object_create(engine->i915, PAGE_ALIGN(size));
+ obj = i915_gem_object_create(engine->i915, CTX_WA_BB_OBJ_SIZE);
if (IS_ERR(obj))
return PTR_ERR(obj);
return err;
}
-static void lrc_destroy_wa_ctx_obj(struct intel_engine_cs *engine)
+static void lrc_destroy_wa_ctx(struct intel_engine_cs *engine)
{
i915_vma_unpin_and_release(&engine->wa_ctx.vma);
}
+typedef u32 *(*wa_bb_func_t)(struct intel_engine_cs *engine, u32 *batch);
+
static int intel_init_workaround_bb(struct intel_engine_cs *engine)
{
struct i915_ctx_workarounds *wa_ctx = &engine->wa_ctx;
- uint32_t *batch;
- uint32_t offset;
+ struct i915_wa_ctx_bb *wa_bb[2] = { &wa_ctx->indirect_ctx,
+ &wa_ctx->per_ctx };
+ wa_bb_func_t wa_bb_fn[2];
struct page *page;
+ void *batch, *batch_ptr;
+ unsigned int i;
int ret;
- WARN_ON(engine->id != RCS);
+ if (WARN_ON(engine->id != RCS || !engine->scratch))
+ return -EINVAL;
- /* update this when WA for higher Gen are added */
- if (INTEL_GEN(engine->i915) > 9) {
- DRM_ERROR("WA batch buffer is not initialized for Gen%d\n",
- INTEL_GEN(engine->i915));
+ switch (INTEL_GEN(engine->i915)) {
+ case 9:
+ wa_bb_fn[0] = gen9_init_indirectctx_bb;
+ wa_bb_fn[1] = gen9_init_perctx_bb;
+ break;
+ case 8:
+ wa_bb_fn[0] = gen8_init_indirectctx_bb;
+ wa_bb_fn[1] = gen8_init_perctx_bb;
+ break;
+ default:
+ MISSING_CASE(INTEL_GEN(engine->i915));
return 0;
}
- /* some WA perform writes to scratch page, ensure it is valid */
- if (!engine->scratch) {
- DRM_ERROR("scratch page not allocated for %s\n", engine->name);
- return -EINVAL;
- }
-
- ret = lrc_setup_wa_ctx_obj(engine, PAGE_SIZE);
+ ret = lrc_setup_wa_ctx(engine);
if (ret) {
DRM_DEBUG_DRIVER("Failed to setup context WA page: %d\n", ret);
return ret;
}
page = i915_gem_object_get_dirty_page(wa_ctx->vma->obj, 0);
- batch = kmap_atomic(page);
- offset = 0;
-
- if (IS_GEN8(engine->i915)) {
- ret = gen8_init_indirectctx_bb(engine,
- &wa_ctx->indirect_ctx,
- batch,
- &offset);
- if (ret)
- goto out;
+ batch = batch_ptr = kmap_atomic(page);
- ret = gen8_init_perctx_bb(engine,
- &wa_ctx->per_ctx,
- batch,
- &offset);
- if (ret)
- goto out;
- } else if (IS_GEN9(engine->i915)) {
- ret = gen9_init_indirectctx_bb(engine,
- &wa_ctx->indirect_ctx,
- batch,
- &offset);
- if (ret)
- goto out;
-
- ret = gen9_init_perctx_bb(engine,
- &wa_ctx->per_ctx,
- batch,
- &offset);
- if (ret)
- goto out;
+ /*
+ * Emit the two workaround batch buffers, recording the offset from the
+ * start of the workaround batch buffer object for each and their
+ * respective sizes.
+ */
+ for (i = 0; i < ARRAY_SIZE(wa_bb_fn); i++) {
+ wa_bb[i]->offset = batch_ptr - batch;
+ if (WARN_ON(!IS_ALIGNED(wa_bb[i]->offset, CACHELINE_BYTES))) {
+ ret = -EINVAL;
+ break;
+ }
+ batch_ptr = wa_bb_fn[i](engine, batch_ptr);
+ wa_bb[i]->size = batch_ptr - (batch + wa_bb[i]->offset);
}
-out:
+ BUG_ON(batch_ptr - batch > CTX_WA_BB_OBJ_SIZE);
+
kunmap_atomic(batch);
if (ret)
- lrc_destroy_wa_ctx_obj(engine);
+ lrc_destroy_wa_ctx(engine);
return ret;
}
+static u32 port_seqno(struct execlist_port *port)
+{
+ return port->request ? port->request->global_seqno : 0;
+}
+
static int gen8_init_common_ring(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
I915_WRITE(RING_HWSTAM(engine->mmio_base), 0xffffffff);
I915_WRITE(RING_MODE_GEN7(engine),
- _MASKED_BIT_DISABLE(GFX_REPLAY_MODE) |
_MASKED_BIT_ENABLE(GFX_RUN_LIST_ENABLE));
I915_WRITE(RING_HWS_PGA(engine->mmio_base),
engine->status_page.ggtt_offset);
DRM_DEBUG_DRIVER("Execlists enabled for %s\n", engine->name);
/* After a GPU reset, we may have requests to replay */
- if (!execlists_elsp_idle(engine)) {
+ clear_bit(ENGINE_IRQ_EXECLIST, &engine->irq_posted);
+ if (!i915.enable_guc_submission && !execlists_elsp_idle(engine)) {
+ DRM_DEBUG_DRIVER("Restarting %s from requests [0x%x, 0x%x]\n",
+ engine->name,
+ port_seqno(&engine->execlist_port[0]),
+ port_seqno(&engine->execlist_port[1]));
engine->execlist_port[0].count = 0;
engine->execlist_port[1].count = 0;
execlists_submit_ports(engine);
static void reset_common_ring(struct intel_engine_cs *engine,
struct drm_i915_gem_request *request)
{
- struct drm_i915_private *dev_priv = engine->i915;
struct execlist_port *port = engine->execlist_port;
struct intel_context *ce;
request->ring->last_retired_head = -1;
intel_ring_update_space(request->ring);
- if (i915.enable_guc_submission)
- return;
-
/* Catch up with any missed context-switch interrupts */
- I915_WRITE(RING_CONTEXT_STATUS_PTR(engine), _MASKED_FIELD(0xffff, 0));
if (request->ctx != port[0].request->ctx) {
i915_gem_request_put(port[0].request);
port[0] = port[1];
/* Reset WaIdleLiteRestore:bdw,skl as well */
request->tail = request->wa_tail - WA_TAIL_DWORDS * sizeof(u32);
+ GEM_BUG_ON(!IS_ALIGNED(request->tail, 8));
}
static int intel_logical_ring_emit_pdps(struct drm_i915_gem_request *req)
{
struct i915_hw_ppgtt *ppgtt = req->ctx->ppgtt;
- struct intel_ring *ring = req->ring;
struct intel_engine_cs *engine = req->engine;
- const int num_lri_cmds = GEN8_LEGACY_PDPES * 2;
- int i, ret;
+ const int num_lri_cmds = GEN8_3LVL_PDPES * 2;
+ u32 *cs;
+ int i;
- ret = intel_ring_begin(req, num_lri_cmds * 2 + 2);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, num_lri_cmds * 2 + 2);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(num_lri_cmds));
- for (i = GEN8_LEGACY_PDPES - 1; i >= 0; i--) {
+ *cs++ = MI_LOAD_REGISTER_IMM(num_lri_cmds);
+ for (i = GEN8_3LVL_PDPES - 1; i >= 0; i--) {
const dma_addr_t pd_daddr = i915_page_dir_dma_addr(ppgtt, i);
- intel_ring_emit_reg(ring, GEN8_RING_PDP_UDW(engine, i));
- intel_ring_emit(ring, upper_32_bits(pd_daddr));
- intel_ring_emit_reg(ring, GEN8_RING_PDP_LDW(engine, i));
- intel_ring_emit(ring, lower_32_bits(pd_daddr));
+ *cs++ = i915_mmio_reg_offset(GEN8_RING_PDP_UDW(engine, i));
+ *cs++ = upper_32_bits(pd_daddr);
+ *cs++ = i915_mmio_reg_offset(GEN8_RING_PDP_LDW(engine, i));
+ *cs++ = lower_32_bits(pd_daddr);
}
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
static int gen8_emit_bb_start(struct drm_i915_gem_request *req,
u64 offset, u32 len,
- unsigned int dispatch_flags)
+ const unsigned int flags)
{
- struct intel_ring *ring = req->ring;
- bool ppgtt = !(dispatch_flags & I915_DISPATCH_SECURE);
+ u32 *cs;
int ret;
/* Don't rely in hw updating PDPs, specially in lite-restore.
* not idle). PML4 is allocated during ppgtt init so this is
* not needed in 48-bit.*/
if (req->ctx->ppgtt &&
- (intel_engine_flag(req->engine) & req->ctx->ppgtt->pd_dirty_rings)) {
- if (!USES_FULL_48BIT_PPGTT(req->i915) &&
- !intel_vgpu_active(req->i915)) {
- ret = intel_logical_ring_emit_pdps(req);
- if (ret)
- return ret;
- }
+ (intel_engine_flag(req->engine) & req->ctx->ppgtt->pd_dirty_rings) &&
+ !i915_vm_is_48bit(&req->ctx->ppgtt->base) &&
+ !intel_vgpu_active(req->i915)) {
+ ret = intel_logical_ring_emit_pdps(req);
+ if (ret)
+ return ret;
req->ctx->ppgtt->pd_dirty_rings &= ~intel_engine_flag(req->engine);
}
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
/* FIXME(BDW): Address space and security selectors. */
- intel_ring_emit(ring, MI_BATCH_BUFFER_START_GEN8 |
- (ppgtt<<8) |
- (dispatch_flags & I915_DISPATCH_RS ?
- MI_BATCH_RESOURCE_STREAMER : 0));
- intel_ring_emit(ring, lower_32_bits(offset));
- intel_ring_emit(ring, upper_32_bits(offset));
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = MI_BATCH_BUFFER_START_GEN8 |
+ (flags & I915_DISPATCH_SECURE ? 0 : BIT(8)) |
+ (flags & I915_DISPATCH_RS ? MI_BATCH_RESOURCE_STREAMER : 0);
+ *cs++ = lower_32_bits(offset);
+ *cs++ = upper_32_bits(offset);
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
static int gen8_emit_flush(struct drm_i915_gem_request *request, u32 mode)
{
- struct intel_ring *ring = request->ring;
- u32 cmd;
- int ret;
+ u32 cmd, *cs;
- ret = intel_ring_begin(request, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(request, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
cmd = MI_FLUSH_DW + 1;
cmd |= MI_INVALIDATE_BSD;
}
- intel_ring_emit(ring, cmd);
- intel_ring_emit(ring,
- I915_GEM_HWS_SCRATCH_ADDR |
- MI_FLUSH_DW_USE_GTT);
- intel_ring_emit(ring, 0); /* upper addr */
- intel_ring_emit(ring, 0); /* value */
- intel_ring_advance(ring);
+ *cs++ = cmd;
+ *cs++ = I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT;
+ *cs++ = 0; /* upper addr */
+ *cs++ = 0; /* value */
+ intel_ring_advance(request, cs);
return 0;
}
static int gen8_emit_flush_render(struct drm_i915_gem_request *request,
u32 mode)
{
- struct intel_ring *ring = request->ring;
struct intel_engine_cs *engine = request->engine;
u32 scratch_addr =
i915_ggtt_offset(engine->scratch) + 2 * CACHELINE_BYTES;
bool vf_flush_wa = false, dc_flush_wa = false;
- u32 flags = 0;
- int ret;
+ u32 *cs, flags = 0;
int len;
flags |= PIPE_CONTROL_CS_STALL;
if (dc_flush_wa)
len += 12;
- ret = intel_ring_begin(request, len);
- if (ret)
- return ret;
+ cs = intel_ring_begin(request, len);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- if (vf_flush_wa) {
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- }
+ if (vf_flush_wa)
+ cs = gen8_emit_pipe_control(cs, 0, 0);
- if (dc_flush_wa) {
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
- intel_ring_emit(ring, PIPE_CONTROL_DC_FLUSH_ENABLE);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- }
+ if (dc_flush_wa)
+ cs = gen8_emit_pipe_control(cs, PIPE_CONTROL_DC_FLUSH_ENABLE,
+ 0);
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
- intel_ring_emit(ring, flags);
- intel_ring_emit(ring, scratch_addr);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
-
- if (dc_flush_wa) {
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
- intel_ring_emit(ring, PIPE_CONTROL_CS_STALL);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- }
+ cs = gen8_emit_pipe_control(cs, flags, scratch_addr);
- intel_ring_advance(ring);
+ if (dc_flush_wa)
+ cs = gen8_emit_pipe_control(cs, PIPE_CONTROL_CS_STALL, 0);
- return 0;
-}
+ intel_ring_advance(request, cs);
-static void bxt_a_seqno_barrier(struct intel_engine_cs *engine)
-{
- /*
- * On BXT A steppings there is a HW coherency issue whereby the
- * MI_STORE_DATA_IMM storing the completed request's seqno
- * occasionally doesn't invalidate the CPU cache. Work around this by
- * clflushing the corresponding cacheline whenever the caller wants
- * the coherency to be guaranteed. Note that this cacheline is known
- * to be clean at this point, since we only write it in
- * bxt_a_set_seqno(), where we also do a clflush after the write. So
- * this clflush in practice becomes an invalidate operation.
- */
- intel_flush_status_page(engine, I915_GEM_HWS_INDEX);
+ return 0;
}
/*
* used as a workaround for not being allowed to do lite
* restore with HEAD==TAIL (WaIdleLiteRestore).
*/
-static void gen8_emit_wa_tail(struct drm_i915_gem_request *request, u32 *out)
+static void gen8_emit_wa_tail(struct drm_i915_gem_request *request, u32 *cs)
{
- *out++ = MI_NOOP;
- *out++ = MI_NOOP;
- request->wa_tail = intel_ring_offset(request->ring, out);
+ *cs++ = MI_NOOP;
+ *cs++ = MI_NOOP;
+ request->wa_tail = intel_ring_offset(request, cs);
}
-static void gen8_emit_breadcrumb(struct drm_i915_gem_request *request,
- u32 *out)
+static void gen8_emit_breadcrumb(struct drm_i915_gem_request *request, u32 *cs)
{
/* w/a: bit 5 needs to be zero for MI_FLUSH_DW address. */
BUILD_BUG_ON(I915_GEM_HWS_INDEX_ADDR & (1 << 5));
- *out++ = (MI_FLUSH_DW + 1) | MI_FLUSH_DW_OP_STOREDW;
- *out++ = intel_hws_seqno_address(request->engine) | MI_FLUSH_DW_USE_GTT;
- *out++ = 0;
- *out++ = request->global_seqno;
- *out++ = MI_USER_INTERRUPT;
- *out++ = MI_NOOP;
- request->tail = intel_ring_offset(request->ring, out);
+ *cs++ = (MI_FLUSH_DW + 1) | MI_FLUSH_DW_OP_STOREDW;
+ *cs++ = intel_hws_seqno_address(request->engine) | MI_FLUSH_DW_USE_GTT;
+ *cs++ = 0;
+ *cs++ = request->global_seqno;
+ *cs++ = MI_USER_INTERRUPT;
+ *cs++ = MI_NOOP;
+ request->tail = intel_ring_offset(request, cs);
+ GEM_BUG_ON(!IS_ALIGNED(request->tail, 8));
- gen8_emit_wa_tail(request, out);
+ gen8_emit_wa_tail(request, cs);
}
static const int gen8_emit_breadcrumb_sz = 6 + WA_TAIL_DWORDS;
static void gen8_emit_breadcrumb_render(struct drm_i915_gem_request *request,
- u32 *out)
+ u32 *cs)
{
/* We're using qword write, seqno should be aligned to 8 bytes. */
BUILD_BUG_ON(I915_GEM_HWS_INDEX & 1);
* need a prior CS_STALL, which is emitted by the flush
* following the batch.
*/
- *out++ = GFX_OP_PIPE_CONTROL(6);
- *out++ = (PIPE_CONTROL_GLOBAL_GTT_IVB |
- PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_QW_WRITE);
- *out++ = intel_hws_seqno_address(request->engine);
- *out++ = 0;
- *out++ = request->global_seqno;
+ *cs++ = GFX_OP_PIPE_CONTROL(6);
+ *cs++ = PIPE_CONTROL_GLOBAL_GTT_IVB | PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_QW_WRITE;
+ *cs++ = intel_hws_seqno_address(request->engine);
+ *cs++ = 0;
+ *cs++ = request->global_seqno;
/* We're thrashing one dword of HWS. */
- *out++ = 0;
- *out++ = MI_USER_INTERRUPT;
- *out++ = MI_NOOP;
- request->tail = intel_ring_offset(request->ring, out);
+ *cs++ = 0;
+ *cs++ = MI_USER_INTERRUPT;
+ *cs++ = MI_NOOP;
+ request->tail = intel_ring_offset(request, cs);
+ GEM_BUG_ON(!IS_ALIGNED(request->tail, 8));
- gen8_emit_wa_tail(request, out);
+ gen8_emit_wa_tail(request, cs);
}
static const int gen8_emit_breadcrumb_render_sz = 8 + WA_TAIL_DWORDS;
{
int ret;
- ret = intel_logical_ring_workarounds_emit(req);
+ ret = intel_ring_workarounds_emit(req);
if (ret)
return ret;
intel_engine_cleanup_common(engine);
- lrc_destroy_wa_ctx_obj(engine);
+ lrc_destroy_wa_ctx(engine);
engine->i915 = NULL;
dev_priv->engine[engine->id] = NULL;
kfree(engine);
}
-void intel_execlists_enable_submission(struct drm_i915_private *dev_priv)
+static void execlists_set_default_submission(struct intel_engine_cs *engine)
{
- struct intel_engine_cs *engine;
- enum intel_engine_id id;
-
- for_each_engine(engine, dev_priv, id) {
- engine->submit_request = execlists_submit_request;
- engine->schedule = execlists_schedule;
- }
+ engine->submit_request = execlists_submit_request;
+ engine->schedule = execlists_schedule;
}
static void
engine->emit_flush = gen8_emit_flush;
engine->emit_breadcrumb = gen8_emit_breadcrumb;
engine->emit_breadcrumb_sz = gen8_emit_breadcrumb_sz;
- engine->submit_request = execlists_submit_request;
- engine->schedule = execlists_schedule;
+
+ engine->set_default_submission = execlists_set_default_submission;
engine->irq_enable = gen8_logical_ring_enable_irq;
engine->irq_disable = gen8_logical_ring_disable_irq;
engine->emit_bb_start = gen8_emit_bb_start;
- if (IS_BXT_REVID(engine->i915, 0, BXT_REVID_A1))
- engine->irq_seqno_barrier = bxt_a_seqno_barrier;
}
static inline void
tasklet_init(&engine->irq_tasklet,
intel_lrc_irq_handler, (unsigned long)engine);
- logical_ring_init_platform_invariants(engine);
logical_ring_default_vfuncs(engine);
logical_ring_default_irqs(engine);
}
return indirect_ctx_offset;
}
-static void execlists_init_reg_state(u32 *reg_state,
+static void execlists_init_reg_state(u32 *regs,
struct i915_gem_context *ctx,
struct intel_engine_cs *engine,
struct intel_ring *ring)
{
struct drm_i915_private *dev_priv = engine->i915;
struct i915_hw_ppgtt *ppgtt = ctx->ppgtt ?: dev_priv->mm.aliasing_ppgtt;
+ u32 base = engine->mmio_base;
+ bool rcs = engine->id == RCS;
+
+ /* A context is actually a big batch buffer with several
+ * MI_LOAD_REGISTER_IMM commands followed by (reg, value) pairs. The
+ * values we are setting here are only for the first context restore:
+ * on a subsequent save, the GPU will recreate this batchbuffer with new
+ * values (including all the missing MI_LOAD_REGISTER_IMM commands that
+ * we are not initializing here).
+ */
+ regs[CTX_LRI_HEADER_0] = MI_LOAD_REGISTER_IMM(rcs ? 14 : 11) |
+ MI_LRI_FORCE_POSTED;
+
+ CTX_REG(regs, CTX_CONTEXT_CONTROL, RING_CONTEXT_CONTROL(engine),
+ _MASKED_BIT_ENABLE(CTX_CTRL_INHIBIT_SYN_CTX_SWITCH |
+ CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT |
+ (HAS_RESOURCE_STREAMER(dev_priv) ?
+ CTX_CTRL_RS_CTX_ENABLE : 0)));
+ CTX_REG(regs, CTX_RING_HEAD, RING_HEAD(base), 0);
+ CTX_REG(regs, CTX_RING_TAIL, RING_TAIL(base), 0);
+ CTX_REG(regs, CTX_RING_BUFFER_START, RING_START(base), 0);
+ CTX_REG(regs, CTX_RING_BUFFER_CONTROL, RING_CTL(base),
+ RING_CTL_SIZE(ring->size) | RING_VALID);
+ CTX_REG(regs, CTX_BB_HEAD_U, RING_BBADDR_UDW(base), 0);
+ CTX_REG(regs, CTX_BB_HEAD_L, RING_BBADDR(base), 0);
+ CTX_REG(regs, CTX_BB_STATE, RING_BBSTATE(base), RING_BB_PPGTT);
+ CTX_REG(regs, CTX_SECOND_BB_HEAD_U, RING_SBBADDR_UDW(base), 0);
+ CTX_REG(regs, CTX_SECOND_BB_HEAD_L, RING_SBBADDR(base), 0);
+ CTX_REG(regs, CTX_SECOND_BB_STATE, RING_SBBSTATE(base), 0);
+ if (rcs) {
+ CTX_REG(regs, CTX_BB_PER_CTX_PTR, RING_BB_PER_CTX_PTR(base), 0);
+ CTX_REG(regs, CTX_RCS_INDIRECT_CTX, RING_INDIRECT_CTX(base), 0);
+ CTX_REG(regs, CTX_RCS_INDIRECT_CTX_OFFSET,
+ RING_INDIRECT_CTX_OFFSET(base), 0);
- /* A context is actually a big batch buffer with several MI_LOAD_REGISTER_IMM
- * commands followed by (reg, value) pairs. The values we are setting here are
- * only for the first context restore: on a subsequent save, the GPU will
- * recreate this batchbuffer with new values (including all the missing
- * MI_LOAD_REGISTER_IMM commands that we are not initializing here). */
- reg_state[CTX_LRI_HEADER_0] =
- MI_LOAD_REGISTER_IMM(engine->id == RCS ? 14 : 11) | MI_LRI_FORCE_POSTED;
- ASSIGN_CTX_REG(reg_state, CTX_CONTEXT_CONTROL,
- RING_CONTEXT_CONTROL(engine),
- _MASKED_BIT_ENABLE(CTX_CTRL_INHIBIT_SYN_CTX_SWITCH |
- CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT |
- (HAS_RESOURCE_STREAMER(dev_priv) ?
- CTX_CTRL_RS_CTX_ENABLE : 0)));
- ASSIGN_CTX_REG(reg_state, CTX_RING_HEAD, RING_HEAD(engine->mmio_base),
- 0);
- ASSIGN_CTX_REG(reg_state, CTX_RING_TAIL, RING_TAIL(engine->mmio_base),
- 0);
- ASSIGN_CTX_REG(reg_state, CTX_RING_BUFFER_START,
- RING_START(engine->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_RING_BUFFER_CONTROL,
- RING_CTL(engine->mmio_base),
- RING_CTL_SIZE(ring->size) | RING_VALID);
- ASSIGN_CTX_REG(reg_state, CTX_BB_HEAD_U,
- RING_BBADDR_UDW(engine->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_BB_HEAD_L,
- RING_BBADDR(engine->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_BB_STATE,
- RING_BBSTATE(engine->mmio_base),
- RING_BB_PPGTT);
- ASSIGN_CTX_REG(reg_state, CTX_SECOND_BB_HEAD_U,
- RING_SBBADDR_UDW(engine->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_SECOND_BB_HEAD_L,
- RING_SBBADDR(engine->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_SECOND_BB_STATE,
- RING_SBBSTATE(engine->mmio_base), 0);
- if (engine->id == RCS) {
- ASSIGN_CTX_REG(reg_state, CTX_BB_PER_CTX_PTR,
- RING_BB_PER_CTX_PTR(engine->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_RCS_INDIRECT_CTX,
- RING_INDIRECT_CTX(engine->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_RCS_INDIRECT_CTX_OFFSET,
- RING_INDIRECT_CTX_OFFSET(engine->mmio_base), 0);
if (engine->wa_ctx.vma) {
struct i915_ctx_workarounds *wa_ctx = &engine->wa_ctx;
u32 ggtt_offset = i915_ggtt_offset(wa_ctx->vma);
- reg_state[CTX_RCS_INDIRECT_CTX+1] =
- (ggtt_offset + wa_ctx->indirect_ctx.offset * sizeof(uint32_t)) |
- (wa_ctx->indirect_ctx.size / CACHELINE_DWORDS);
+ regs[CTX_RCS_INDIRECT_CTX + 1] =
+ (ggtt_offset + wa_ctx->indirect_ctx.offset) |
+ (wa_ctx->indirect_ctx.size / CACHELINE_BYTES);
- reg_state[CTX_RCS_INDIRECT_CTX_OFFSET+1] =
+ regs[CTX_RCS_INDIRECT_CTX_OFFSET + 1] =
intel_lr_indirect_ctx_offset(engine) << 6;
- reg_state[CTX_BB_PER_CTX_PTR+1] =
- (ggtt_offset + wa_ctx->per_ctx.offset * sizeof(uint32_t)) |
- 0x01;
+ regs[CTX_BB_PER_CTX_PTR + 1] =
+ (ggtt_offset + wa_ctx->per_ctx.offset) | 0x01;
}
}
- reg_state[CTX_LRI_HEADER_1] = MI_LOAD_REGISTER_IMM(9) | MI_LRI_FORCE_POSTED;
- ASSIGN_CTX_REG(reg_state, CTX_CTX_TIMESTAMP,
- RING_CTX_TIMESTAMP(engine->mmio_base), 0);
+
+ regs[CTX_LRI_HEADER_1] = MI_LOAD_REGISTER_IMM(9) | MI_LRI_FORCE_POSTED;
+
+ CTX_REG(regs, CTX_CTX_TIMESTAMP, RING_CTX_TIMESTAMP(base), 0);
/* PDP values well be assigned later if needed */
- ASSIGN_CTX_REG(reg_state, CTX_PDP3_UDW, GEN8_RING_PDP_UDW(engine, 3),
- 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP3_LDW, GEN8_RING_PDP_LDW(engine, 3),
- 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP2_UDW, GEN8_RING_PDP_UDW(engine, 2),
- 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP2_LDW, GEN8_RING_PDP_LDW(engine, 2),
- 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP1_UDW, GEN8_RING_PDP_UDW(engine, 1),
- 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP1_LDW, GEN8_RING_PDP_LDW(engine, 1),
- 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP0_UDW, GEN8_RING_PDP_UDW(engine, 0),
- 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP0_LDW, GEN8_RING_PDP_LDW(engine, 0),
- 0);
-
- if (ppgtt && USES_FULL_48BIT_PPGTT(ppgtt->base.dev)) {
+ CTX_REG(regs, CTX_PDP3_UDW, GEN8_RING_PDP_UDW(engine, 3), 0);
+ CTX_REG(regs, CTX_PDP3_LDW, GEN8_RING_PDP_LDW(engine, 3), 0);
+ CTX_REG(regs, CTX_PDP2_UDW, GEN8_RING_PDP_UDW(engine, 2), 0);
+ CTX_REG(regs, CTX_PDP2_LDW, GEN8_RING_PDP_LDW(engine, 2), 0);
+ CTX_REG(regs, CTX_PDP1_UDW, GEN8_RING_PDP_UDW(engine, 1), 0);
+ CTX_REG(regs, CTX_PDP1_LDW, GEN8_RING_PDP_LDW(engine, 1), 0);
+ CTX_REG(regs, CTX_PDP0_UDW, GEN8_RING_PDP_UDW(engine, 0), 0);
+ CTX_REG(regs, CTX_PDP0_LDW, GEN8_RING_PDP_LDW(engine, 0), 0);
+
+ if (ppgtt && i915_vm_is_48bit(&ppgtt->base)) {
/* 64b PPGTT (48bit canonical)
* PDP0_DESCRIPTOR contains the base address to PML4 and
* other PDP Descriptors are ignored.
*/
- ASSIGN_CTX_PML4(ppgtt, reg_state);
+ ASSIGN_CTX_PML4(ppgtt, regs);
}
- if (engine->id == RCS) {
- reg_state[CTX_LRI_HEADER_2] = MI_LOAD_REGISTER_IMM(1);
- ASSIGN_CTX_REG(reg_state, CTX_R_PWR_CLK_STATE, GEN8_R_PWR_CLK_STATE,
- make_rpcs(dev_priv));
+ if (rcs) {
+ regs[CTX_LRI_HEADER_2] = MI_LOAD_REGISTER_IMM(1);
+ CTX_REG(regs, CTX_R_PWR_CLK_STATE, GEN8_R_PWR_CLK_STATE,
+ make_rpcs(dev_priv));
}
}
int logical_render_ring_init(struct intel_engine_cs *engine);
int logical_xcs_ring_init(struct intel_engine_cs *engine);
-int intel_engines_init(struct drm_i915_private *dev_priv);
-
/* Logical Ring Contexts */
/* One extra page is added before LRC for GuC as shared data */
/* Execlists */
int intel_sanitize_enable_execlists(struct drm_i915_private *dev_priv,
int enable_execlists);
-void intel_execlists_enable_submission(struct drm_i915_private *dev_priv);
-bool intel_execlists_idle(struct drm_i915_private *dev_priv);
#endif /* _INTEL_LRC_H_ */
struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);
unsigned long start = jiffies;
- if (!lspcon->desc_valid)
- return;
-
while (1) {
- struct intel_dp_desc desc;
-
- /*
- * The w/a only applies in PCON mode and we don't expect any
- * AUX errors.
- */
- if (!__intel_dp_read_desc(intel_dp, &desc))
- return;
-
- if (intel_digital_port_connected(dev_priv, dig_port) &&
- !memcmp(&intel_dp->desc, &desc, sizeof(desc))) {
+ if (intel_digital_port_connected(dev_priv, dig_port)) {
DRM_DEBUG_KMS("LSPCON recovering in PCON mode after %u ms\n",
jiffies_to_msecs(jiffies - start));
return;
return false;
}
- lspcon->desc_valid = intel_dp_read_desc(dp);
+ intel_dp_read_desc(dp);
DRM_DEBUG_KMS("Success: LSPCON init\n");
return true;
struct drm_device *dev = encoder->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(&encoder->base);
- enum intel_display_power_domain power_domain;
u32 tmp;
bool ret;
- power_domain = intel_display_port_power_domain(encoder);
- if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
+ if (!intel_display_power_get_if_enabled(dev_priv,
+ encoder->power_domain))
return false;
ret = false;
ret = true;
out:
- intel_display_power_put(dev_priv, power_domain);
+ intel_display_power_put(dev_priv, encoder->power_domain);
return ret;
}
intel_connector_attach_encoder(intel_connector, intel_encoder);
intel_encoder->type = INTEL_OUTPUT_LVDS;
+ intel_encoder->power_domain = POWER_DOMAIN_PORT_OTHER;
intel_encoder->port = PORT_NONE;
intel_encoder->cloneable = 0;
if (HAS_PCH_SPLIT(dev_priv))
{
bool result = false;
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ if (IS_GEN9_BC(dev_priv)) {
table->size = ARRAY_SIZE(skylake_mocs_table);
table->table = skylake_mocs_table;
result = true;
"Platform that should have a MOCS table does not.\n");
}
- /* WaDisableSkipCaching:skl,bxt,kbl */
+ /* WaDisableSkipCaching:skl,bxt,kbl,glk */
if (IS_GEN9(dev_priv)) {
int i;
static int emit_mocs_control_table(struct drm_i915_gem_request *req,
const struct drm_i915_mocs_table *table)
{
- struct intel_ring *ring = req->ring;
enum intel_engine_id engine = req->engine->id;
unsigned int index;
- int ret;
+ u32 *cs;
if (WARN_ON(table->size > GEN9_NUM_MOCS_ENTRIES))
return -ENODEV;
- ret = intel_ring_begin(req, 2 + 2 * GEN9_NUM_MOCS_ENTRIES);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 2 + 2 * GEN9_NUM_MOCS_ENTRIES);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES));
+ *cs++ = MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES);
for (index = 0; index < table->size; index++) {
- intel_ring_emit_reg(ring, mocs_register(engine, index));
- intel_ring_emit(ring, table->table[index].control_value);
+ *cs++ = i915_mmio_reg_offset(mocs_register(engine, index));
+ *cs++ = table->table[index].control_value;
}
/*
* that value to all the used entries.
*/
for (; index < GEN9_NUM_MOCS_ENTRIES; index++) {
- intel_ring_emit_reg(ring, mocs_register(engine, index));
- intel_ring_emit(ring, table->table[0].control_value);
+ *cs++ = i915_mmio_reg_offset(mocs_register(engine, index));
+ *cs++ = table->table[0].control_value;
}
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
static int emit_mocs_l3cc_table(struct drm_i915_gem_request *req,
const struct drm_i915_mocs_table *table)
{
- struct intel_ring *ring = req->ring;
unsigned int i;
- int ret;
+ u32 *cs;
if (WARN_ON(table->size > GEN9_NUM_MOCS_ENTRIES))
return -ENODEV;
- ret = intel_ring_begin(req, 2 + GEN9_NUM_MOCS_ENTRIES);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 2 + GEN9_NUM_MOCS_ENTRIES);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring,
- MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES / 2));
+ *cs++ = MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES / 2);
for (i = 0; i < table->size/2; i++) {
- intel_ring_emit_reg(ring, GEN9_LNCFCMOCS(i));
- intel_ring_emit(ring, l3cc_combine(table, 2*i, 2*i+1));
+ *cs++ = i915_mmio_reg_offset(GEN9_LNCFCMOCS(i));
+ *cs++ = l3cc_combine(table, 2 * i, 2 * i + 1);
}
if (table->size & 0x01) {
/* Odd table size - 1 left over */
- intel_ring_emit_reg(ring, GEN9_LNCFCMOCS(i));
- intel_ring_emit(ring, l3cc_combine(table, 2*i, 0));
+ *cs++ = i915_mmio_reg_offset(GEN9_LNCFCMOCS(i));
+ *cs++ = l3cc_combine(table, 2 * i, 0);
i++;
}
* they are reserved by the hardware.
*/
for (; i < GEN9_NUM_MOCS_ENTRIES / 2; i++) {
- intel_ring_emit_reg(ring, GEN9_LNCFCMOCS(i));
- intel_ring_emit(ring, l3cc_combine(table, 0, 0));
+ *cs++ = i915_mmio_reg_offset(GEN9_LNCFCMOCS(i));
+ *cs++ = l3cc_combine(table, 0, 0);
}
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
static u32 asle_set_backlight(struct drm_i915_private *dev_priv, u32 bclp)
{
struct intel_connector *connector;
+ struct drm_connector_list_iter conn_iter;
struct opregion_asle *asle = dev_priv->opregion.asle;
struct drm_device *dev = &dev_priv->drm;
* only one).
*/
DRM_DEBUG_KMS("updating opregion backlight %d/255\n", bclp);
- for_each_intel_connector(dev, connector)
+ drm_connector_list_iter_begin(dev, &conn_iter);
+ for_each_intel_connector_iter(connector, &conn_iter)
intel_panel_set_backlight_acpi(connector, bclp, 255);
+ drm_connector_list_iter_end(&conn_iter);
asle->cblv = DIV_ROUND_UP(bclp * 100, 255) | ASLE_CBLV_VALID;
drm_modeset_unlock(&dev->mode_config.connection_mutex);
{
struct intel_opregion *opregion = &dev_priv->opregion;
struct intel_connector *connector;
+ struct drm_connector_list_iter conn_iter;
int i = 0, max_outputs;
int display_index[16] = {};
max_outputs = ARRAY_SIZE(opregion->acpi->didl) +
ARRAY_SIZE(opregion->acpi->did2);
- for_each_intel_connector(&dev_priv->drm, connector) {
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
+ for_each_intel_connector_iter(connector, &conn_iter) {
u32 device_id, type;
device_id = acpi_display_type(connector);
set_did(opregion, i, device_id);
i++;
}
+ drm_connector_list_iter_end(&conn_iter);
DRM_DEBUG_KMS("%d outputs detected\n", i);
{
struct intel_opregion *opregion = &dev_priv->opregion;
struct intel_connector *connector;
+ struct drm_connector_list_iter conn_iter;
int i = 0;
/*
* Note that internal panels should be at the front of the connector
* list already, ensuring they're not left out.
*/
- for_each_intel_connector(&dev_priv->drm, connector) {
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
+ for_each_intel_connector_iter(connector, &conn_iter) {
if (i >= ARRAY_SIZE(opregion->acpi->cadl))
break;
opregion->acpi->cadl[i++] = connector->acpi_device_id;
}
+ drm_connector_list_iter_end(&conn_iter);
/* If fewer than 8 active devices, the list must be null terminated */
if (i < ARRAY_SIZE(opregion->acpi->cadl))
return -ENODEV;
}
- /*
- * FIXME On Dell XPS 13 9350 the OpRegion panel type (0) gives us
- * low vswing for eDP, whereas the VBT panel type (2) gives us normal
- * vswing instead. Low vswing results in some display flickers, so
- * let's simply ignore the OpRegion panel type on SKL for now.
- */
- if (IS_SKYLAKE(dev_priv)) {
- DRM_DEBUG_KMS("Ignoring OpRegion panel type (%d)\n", ret - 1);
- return -ENODEV;
- }
-
return ret - 1;
}
{
struct drm_i915_private *dev_priv = overlay->i915;
struct drm_i915_gem_request *req;
- struct intel_ring *ring;
- int ret;
+ u32 *cs;
WARN_ON(overlay->active);
WARN_ON(IS_I830(dev_priv) && !(dev_priv->quirks & QUIRK_PIPEA_FORCE));
if (IS_ERR(req))
return PTR_ERR(req);
- ret = intel_ring_begin(req, 4);
- if (ret) {
- i915_add_request_no_flush(req);
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs)) {
+ i915_add_request(req);
+ return PTR_ERR(cs);
}
overlay->active = true;
if (IS_I830(dev_priv))
i830_overlay_clock_gating(dev_priv, false);
- ring = req->ring;
- intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_ON);
- intel_ring_emit(ring, overlay->flip_addr | OFC_UPDATE);
- intel_ring_emit(ring, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = MI_OVERLAY_FLIP | MI_OVERLAY_ON;
+ *cs++ = overlay->flip_addr | OFC_UPDATE;
+ *cs++ = MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP;
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return intel_overlay_do_wait_request(overlay, req, NULL);
}
{
struct drm_i915_private *dev_priv = overlay->i915;
struct drm_i915_gem_request *req;
- struct intel_ring *ring;
u32 flip_addr = overlay->flip_addr;
- u32 tmp;
- int ret;
+ u32 tmp, *cs;
WARN_ON(!overlay->active);
if (IS_ERR(req))
return PTR_ERR(req);
- ret = intel_ring_begin(req, 2);
- if (ret) {
- i915_add_request_no_flush(req);
- return ret;
+ cs = intel_ring_begin(req, 2);
+ if (IS_ERR(cs)) {
+ i915_add_request(req);
+ return PTR_ERR(cs);
}
- ring = req->ring;
- intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
- intel_ring_emit(ring, flip_addr);
- intel_ring_advance(ring);
+ *cs++ = MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE;
+ *cs++ = flip_addr;
+ intel_ring_advance(req, cs);
intel_overlay_flip_prepare(overlay, vma);
static int intel_overlay_off(struct intel_overlay *overlay)
{
struct drm_i915_gem_request *req;
- struct intel_ring *ring;
- u32 flip_addr = overlay->flip_addr;
- int ret;
+ u32 *cs, flip_addr = overlay->flip_addr;
WARN_ON(!overlay->active);
if (IS_ERR(req))
return PTR_ERR(req);
- ret = intel_ring_begin(req, 6);
- if (ret) {
- i915_add_request_no_flush(req);
- return ret;
+ cs = intel_ring_begin(req, 6);
+ if (IS_ERR(cs)) {
+ i915_add_request(req);
+ return PTR_ERR(cs);
}
- ring = req->ring;
-
/* wait for overlay to go idle */
- intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
- intel_ring_emit(ring, flip_addr);
- intel_ring_emit(ring, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
+ *cs++ = MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE;
+ *cs++ = flip_addr;
+ *cs++ = MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP;
/* turn overlay off */
- intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_OFF);
- intel_ring_emit(ring, flip_addr);
- intel_ring_emit(ring, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
+ *cs++ = MI_OVERLAY_FLIP | MI_OVERLAY_OFF;
+ *cs++ = flip_addr;
+ *cs++ = MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP;
- intel_ring_advance(ring);
+ intel_ring_advance(req, cs);
intel_overlay_flip_prepare(overlay, NULL);
static int intel_overlay_release_old_vid(struct intel_overlay *overlay)
{
struct drm_i915_private *dev_priv = overlay->i915;
+ u32 *cs;
int ret;
lockdep_assert_held(&dev_priv->drm.struct_mutex);
if (I915_READ(ISR) & I915_OVERLAY_PLANE_FLIP_PENDING_INTERRUPT) {
/* synchronous slowpath */
struct drm_i915_gem_request *req;
- struct intel_ring *ring;
req = alloc_request(overlay);
if (IS_ERR(req))
return PTR_ERR(req);
- ret = intel_ring_begin(req, 2);
- if (ret) {
- i915_add_request_no_flush(req);
- return ret;
+ cs = intel_ring_begin(req, 2);
+ if (IS_ERR(cs)) {
+ i915_add_request(req);
+ return PTR_ERR(cs);
}
- ring = req->ring;
- intel_ring_emit(ring,
- MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP;
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
ret = intel_overlay_do_wait_request(overlay, req,
intel_overlay_release_old_vid_tail);
if (IS_PINEVIEW(dev_priv))
clock = KHz(dev_priv->rawclk_freq);
else
- clock = KHz(dev_priv->cdclk_freq);
+ clock = KHz(dev_priv->cdclk.hw.cdclk);
return DIV_ROUND_CLOSEST(clock, pwm_freq_hz * 32);
}
if (IS_G4X(dev_priv))
clock = KHz(dev_priv->rawclk_freq);
else
- clock = KHz(dev_priv->cdclk_freq);
+ clock = KHz(dev_priv->cdclk.hw.cdclk);
return DIV_ROUND_CLOSEST(clock, pwm_freq_hz * 128);
}
enum pipe pipe;
};
-/* As the drm_debugfs_init() routines are called before dev->dev_private is
- * allocated we need to hook into the minor for release.
- */
-static int drm_add_fake_info_node(struct drm_minor *minor,
- struct dentry *ent, const void *key)
-{
- struct drm_info_node *node;
-
- node = kmalloc(sizeof(*node), GFP_KERNEL);
- if (node == NULL) {
- debugfs_remove(ent);
- return -ENOMEM;
- }
-
- node->minor = minor;
- node->dent = ent;
- node->info_ent = (void *) key;
-
- mutex_lock(&minor->debugfs_lock);
- list_add(&node->list, &minor->debugfs_list);
- mutex_unlock(&minor->debugfs_lock);
-
- return 0;
-}
-
static int i915_pipe_crc_open(struct inode *inode, struct file *filep)
{
struct pipe_crc_info *info = inode->i_private;
static int pipe_crc_data_count(struct intel_pipe_crc *pipe_crc)
{
- assert_spin_locked(&pipe_crc->lock);
+ lockdep_assert_held(&pipe_crc->lock);
return CIRC_CNT(pipe_crc->head, pipe_crc->tail,
INTEL_PIPE_CRC_ENTRIES_NR);
}
},
};
-static int i915_pipe_crc_create(struct dentry *root, struct drm_minor *minor,
- enum pipe pipe)
-{
- struct drm_i915_private *dev_priv = to_i915(minor->dev);
- struct dentry *ent;
- struct pipe_crc_info *info = &i915_pipe_crc_data[pipe];
-
- info->dev_priv = dev_priv;
- ent = debugfs_create_file(info->name, S_IRUGO, root, info,
- &i915_pipe_crc_fops);
- if (!ent)
- return -ENOMEM;
-
- return drm_add_fake_info_node(minor, ent, info);
-}
-
static const char * const pipe_crc_sources[] = {
"none",
"plane1",
int intel_pipe_crc_create(struct drm_minor *minor)
{
- int ret, i;
-
- for (i = 0; i < ARRAY_SIZE(i915_pipe_crc_data); i++) {
- ret = i915_pipe_crc_create(minor->debugfs_root, minor, i);
- if (ret)
- return ret;
- }
-
- return 0;
-}
-
-void intel_pipe_crc_cleanup(struct drm_minor *minor)
-{
+ struct drm_i915_private *dev_priv = to_i915(minor->dev);
+ struct dentry *ent;
int i;
for (i = 0; i < ARRAY_SIZE(i915_pipe_crc_data); i++) {
- struct drm_info_list *info_list =
- (struct drm_info_list *)&i915_pipe_crc_data[i];
+ struct pipe_crc_info *info = &i915_pipe_crc_data[i];
- drm_debugfs_remove_files(info_list, 1, minor);
+ info->dev_priv = dev_priv;
+ ent = debugfs_create_file(info->name, S_IRUGO,
+ minor->debugfs_root, info,
+ &i915_pipe_crc_fops);
+ if (!ent)
+ return -ENOMEM;
}
+
+ return 0;
}
int intel_crtc_set_crc_source(struct drm_crtc *crtc, const char *source_name,
I915_WRITE(GEN8_CONFIG0,
I915_READ(GEN8_CONFIG0) | GEN9_DEFAULT_FIXES);
- /* WaEnableChickenDCPR:skl,bxt,kbl */
+ /* WaEnableChickenDCPR:skl,bxt,kbl,glk */
I915_WRITE(GEN8_CHICKEN_DCPR_1,
I915_READ(GEN8_CHICKEN_DCPR_1) | MASK_WAKEMEM);
/* WaFbcTurnOffFbcWatermark:skl,bxt,kbl */
- /* WaFbcWakeMemOn:skl,bxt,kbl */
+ /* WaFbcWakeMemOn:skl,bxt,kbl,glk */
I915_WRITE(DISP_ARB_CTL, I915_READ(DISP_ARB_CTL) |
DISP_FBC_WM_DIS |
DISP_FBC_MEMORY_WAKE);
* Wa: Backlight PWM may stop in the asserted state, causing backlight
* to stay fully on.
*/
- if (IS_BXT_REVID(dev_priv, BXT_REVID_B0, REVID_FOREVER))
- I915_WRITE(GEN9_CLKGATE_DIS_0, I915_READ(GEN9_CLKGATE_DIS_0) |
- PWM1_GATING_DIS | PWM2_GATING_DIS);
+ I915_WRITE(GEN9_CLKGATE_DIS_0, I915_READ(GEN9_CLKGATE_DIS_0) |
+ PWM1_GATING_DIS | PWM2_GATING_DIS);
+}
+
+static void glk_init_clock_gating(struct drm_i915_private *dev_priv)
+{
+ gen9_init_clock_gating(dev_priv);
+
+ /*
+ * WaDisablePWMClockGating:glk
+ * Backlight PWM may stop in the asserted state, causing backlight
+ * to stay fully on.
+ */
+ I915_WRITE(GEN9_CLKGATE_DIS_0, I915_READ(GEN9_CLKGATE_DIS_0) |
+ PWM1_GATING_DIS | PWM2_GATING_DIS);
+
+ /* WaDDIIOTimeout:glk */
+ if (IS_GLK_REVID(dev_priv, 0, GLK_REVID_A1)) {
+ u32 val = I915_READ(CHICKEN_MISC_2);
+ val &= ~(GLK_CL0_PWR_DOWN |
+ GLK_CL1_PWR_DOWN |
+ GLK_CL2_PWR_DOWN);
+ I915_WRITE(CHICKEN_MISC_2, val);
+ }
+
}
static void i915_pineview_get_mem_freq(struct drm_i915_private *dev_priv)
return false;
}
+ trace_intel_memory_cxsr(dev_priv, was_enabled, enable);
+
DRM_DEBUG_KMS("memory self-refresh is %s (was %s)\n",
enableddisabled(enable),
enableddisabled(was_enabled));
#define VLV_FIFO_START(dsparb, dsparb2, lo_shift, hi_shift) \
((((dsparb) >> (lo_shift)) & 0xff) | ((((dsparb2) >> (hi_shift)) & 0x1) << 8))
-static int vlv_get_fifo_size(struct intel_plane *plane)
+static void vlv_get_fifo_size(struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
- int sprite0_start, sprite1_start, size;
-
- if (plane->id == PLANE_CURSOR)
- return 63;
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct vlv_fifo_state *fifo_state = &crtc_state->wm.vlv.fifo_state;
+ enum pipe pipe = crtc->pipe;
+ int sprite0_start, sprite1_start;
- switch (plane->pipe) {
+ switch (pipe) {
uint32_t dsparb, dsparb2, dsparb3;
case PIPE_A:
dsparb = I915_READ(DSPARB);
sprite1_start = VLV_FIFO_START(dsparb3, dsparb2, 8, 20);
break;
default:
- return 0;
- }
-
- switch (plane->id) {
- case PLANE_PRIMARY:
- size = sprite0_start;
- break;
- case PLANE_SPRITE0:
- size = sprite1_start - sprite0_start;
- break;
- case PLANE_SPRITE1:
- size = 512 - 1 - sprite1_start;
- break;
- default:
- return 0;
+ MISSING_CASE(pipe);
+ return;
}
- DRM_DEBUG_KMS("%s FIFO size: %d\n", plane->base.name, size);
+ fifo_state->plane[PLANE_PRIMARY] = sprite0_start;
+ fifo_state->plane[PLANE_SPRITE0] = sprite1_start - sprite0_start;
+ fifo_state->plane[PLANE_SPRITE1] = 511 - sprite1_start;
+ fifo_state->plane[PLANE_CURSOR] = 63;
- return size;
+ DRM_DEBUG_KMS("Pipe %c FIFO size: %d/%d/%d/%d\n",
+ pipe_name(pipe),
+ fifo_state->plane[PLANE_PRIMARY],
+ fifo_state->plane[PLANE_SPRITE0],
+ fifo_state->plane[PLANE_SPRITE1],
+ fifo_state->plane[PLANE_CURSOR]);
}
static int i9xx_get_fifo_size(struct drm_i915_private *dev_priv, int plane)
enum pipe pipe;
for_each_pipe(dev_priv, pipe) {
+ trace_vlv_wm(intel_get_crtc_for_pipe(dev_priv, pipe), wm);
+
I915_WRITE(VLV_DDL(pipe),
(wm->ddl[pipe].plane[PLANE_CURSOR] << DDL_CURSOR_SHIFT) |
(wm->ddl[pipe].plane[PLANE_SPRITE1] << DDL_SPRITE_SHIFT(1)) |
#undef FW_WM_VLV
-enum vlv_wm_level {
- VLV_WM_LEVEL_PM2,
- VLV_WM_LEVEL_PM5,
- VLV_WM_LEVEL_DDR_DVFS,
-};
-
/* latency must be in 0.1us units. */
static unsigned int vlv_wm_method2(unsigned int pixel_rate,
unsigned int pipe_htotal,
return min_t(int, wm, USHRT_MAX);
}
-static void vlv_compute_fifo(struct intel_crtc *crtc)
+static bool vlv_need_sprite0_fifo_workaround(unsigned int active_planes)
{
- struct drm_device *dev = crtc->base.dev;
- struct vlv_wm_state *wm_state = &crtc->wm_state;
- struct intel_plane *plane;
- unsigned int total_rate = 0;
- const int fifo_size = 512 - 1;
+ return (active_planes & (BIT(PLANE_SPRITE0) |
+ BIT(PLANE_SPRITE1))) == BIT(PLANE_SPRITE1);
+}
+
+static int vlv_compute_fifo(struct intel_crtc_state *crtc_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+ const struct vlv_pipe_wm *raw =
+ &crtc_state->wm.vlv.raw[VLV_WM_LEVEL_PM2];
+ struct vlv_fifo_state *fifo_state = &crtc_state->wm.vlv.fifo_state;
+ unsigned int active_planes = crtc_state->active_planes & ~BIT(PLANE_CURSOR);
+ int num_active_planes = hweight32(active_planes);
+ const int fifo_size = 511;
int fifo_extra, fifo_left = fifo_size;
+ int sprite0_fifo_extra = 0;
+ unsigned int total_rate;
+ enum plane_id plane_id;
- for_each_intel_plane_on_crtc(dev, crtc, plane) {
- struct intel_plane_state *state =
- to_intel_plane_state(plane->base.state);
+ /*
+ * When enabling sprite0 after sprite1 has already been enabled
+ * we tend to get an underrun unless sprite0 already has some
+ * FIFO space allcoated. Hence we always allocate at least one
+ * cacheline for sprite0 whenever sprite1 is enabled.
+ *
+ * All other plane enable sequences appear immune to this problem.
+ */
+ if (vlv_need_sprite0_fifo_workaround(active_planes))
+ sprite0_fifo_extra = 1;
- if (plane->base.type == DRM_PLANE_TYPE_CURSOR)
- continue;
+ total_rate = raw->plane[PLANE_PRIMARY] +
+ raw->plane[PLANE_SPRITE0] +
+ raw->plane[PLANE_SPRITE1] +
+ sprite0_fifo_extra;
- if (state->base.visible) {
- wm_state->num_active_planes++;
- total_rate += state->base.fb->format->cpp[0];
- }
- }
+ if (total_rate > fifo_size)
+ return -EINVAL;
- for_each_intel_plane_on_crtc(dev, crtc, plane) {
- struct intel_plane_state *state =
- to_intel_plane_state(plane->base.state);
- unsigned int rate;
+ if (total_rate == 0)
+ total_rate = 1;
- if (plane->base.type == DRM_PLANE_TYPE_CURSOR) {
- plane->wm.fifo_size = 63;
- continue;
- }
+ for_each_plane_id_on_crtc(crtc, plane_id) {
+ unsigned int rate;
- if (!state->base.visible) {
- plane->wm.fifo_size = 0;
+ if ((active_planes & BIT(plane_id)) == 0) {
+ fifo_state->plane[plane_id] = 0;
continue;
}
- rate = state->base.fb->format->cpp[0];
- plane->wm.fifo_size = fifo_size * rate / total_rate;
- fifo_left -= plane->wm.fifo_size;
+ rate = raw->plane[plane_id];
+ fifo_state->plane[plane_id] = fifo_size * rate / total_rate;
+ fifo_left -= fifo_state->plane[plane_id];
}
- fifo_extra = DIV_ROUND_UP(fifo_left, wm_state->num_active_planes ?: 1);
+ fifo_state->plane[PLANE_SPRITE0] += sprite0_fifo_extra;
+ fifo_left -= sprite0_fifo_extra;
+
+ fifo_state->plane[PLANE_CURSOR] = 63;
+
+ fifo_extra = DIV_ROUND_UP(fifo_left, num_active_planes ?: 1);
/* spread the remainder evenly */
- for_each_intel_plane_on_crtc(dev, crtc, plane) {
+ for_each_plane_id_on_crtc(crtc, plane_id) {
int plane_extra;
if (fifo_left == 0)
break;
- if (plane->base.type == DRM_PLANE_TYPE_CURSOR)
- continue;
-
- /* give it all to the first plane if none are active */
- if (plane->wm.fifo_size == 0 &&
- wm_state->num_active_planes)
+ if ((active_planes & BIT(plane_id)) == 0)
continue;
plane_extra = min(fifo_extra, fifo_left);
- plane->wm.fifo_size += plane_extra;
+ fifo_state->plane[plane_id] += plane_extra;
fifo_left -= plane_extra;
}
- WARN_ON(fifo_left != 0);
+ WARN_ON(active_planes != 0 && fifo_left != 0);
+
+ /* give it all to the first plane if none are active */
+ if (active_planes == 0) {
+ WARN_ON(fifo_left != fifo_size);
+ fifo_state->plane[PLANE_PRIMARY] = fifo_left;
+ }
+
+ return 0;
+}
+
+static int vlv_num_wm_levels(struct drm_i915_private *dev_priv)
+{
+ return dev_priv->wm.max_level + 1;
+}
+
+/* mark all levels starting from 'level' as invalid */
+static void vlv_invalidate_wms(struct intel_crtc *crtc,
+ struct vlv_wm_state *wm_state, int level)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+
+ for (; level < vlv_num_wm_levels(dev_priv); level++) {
+ enum plane_id plane_id;
+
+ for_each_plane_id_on_crtc(crtc, plane_id)
+ wm_state->wm[level].plane[plane_id] = USHRT_MAX;
+
+ wm_state->sr[level].cursor = USHRT_MAX;
+ wm_state->sr[level].plane = USHRT_MAX;
+ }
}
static u16 vlv_invert_wm_value(u16 wm, u16 fifo_size)
return fifo_size - wm;
}
-static void vlv_invert_wms(struct intel_crtc *crtc)
+/*
+ * Starting from 'level' set all higher
+ * levels to 'value' in the "raw" watermarks.
+ */
+static bool vlv_raw_plane_wm_set(struct intel_crtc_state *crtc_state,
+ int level, enum plane_id plane_id, u16 value)
{
- struct vlv_wm_state *wm_state = &crtc->wm_state;
- int level;
+ struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
+ int num_levels = vlv_num_wm_levels(dev_priv);
+ bool dirty = false;
- for (level = 0; level < wm_state->num_levels; level++) {
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- const int sr_fifo_size =
- INTEL_INFO(dev_priv)->num_pipes * 512 - 1;
- struct intel_plane *plane;
+ for (; level < num_levels; level++) {
+ struct vlv_pipe_wm *raw = &crtc_state->wm.vlv.raw[level];
- wm_state->sr[level].plane =
- vlv_invert_wm_value(wm_state->sr[level].plane,
- sr_fifo_size);
- wm_state->sr[level].cursor =
- vlv_invert_wm_value(wm_state->sr[level].cursor,
- 63);
-
- for_each_intel_plane_on_crtc(&dev_priv->drm, crtc, plane) {
- wm_state->wm[level].plane[plane->id] =
- vlv_invert_wm_value(wm_state->wm[level].plane[plane->id],
- plane->wm.fifo_size);
- }
+ dirty |= raw->plane[plane_id] != value;
+ raw->plane[plane_id] = value;
}
+
+ return dirty;
}
-static void vlv_compute_wm(struct intel_crtc *crtc)
+static bool vlv_plane_wm_compute(struct intel_crtc_state *crtc_state,
+ const struct intel_plane_state *plane_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct vlv_wm_state *wm_state = &crtc->wm_state;
- struct intel_plane *plane;
+ struct intel_plane *plane = to_intel_plane(plane_state->base.plane);
+ enum plane_id plane_id = plane->id;
+ int num_levels = vlv_num_wm_levels(to_i915(plane->base.dev));
int level;
+ bool dirty = false;
+
+ if (!plane_state->base.visible) {
+ dirty |= vlv_raw_plane_wm_set(crtc_state, 0, plane_id, 0);
+ goto out;
+ }
+
+ for (level = 0; level < num_levels; level++) {
+ struct vlv_pipe_wm *raw = &crtc_state->wm.vlv.raw[level];
+ int wm = vlv_compute_wm_level(crtc_state, plane_state, level);
+ int max_wm = plane_id == PLANE_CURSOR ? 63 : 511;
- memset(wm_state, 0, sizeof(*wm_state));
+ if (wm > max_wm)
+ break;
+
+ dirty |= raw->plane[plane_id] != wm;
+ raw->plane[plane_id] = wm;
+ }
+
+ /* mark all higher levels as invalid */
+ dirty |= vlv_raw_plane_wm_set(crtc_state, level, plane_id, USHRT_MAX);
+
+out:
+ if (dirty)
+ DRM_DEBUG_KMS("%s wms: [0]=%d,[1]=%d,[2]=%d\n",
+ plane->base.name,
+ crtc_state->wm.vlv.raw[VLV_WM_LEVEL_PM2].plane[plane_id],
+ crtc_state->wm.vlv.raw[VLV_WM_LEVEL_PM5].plane[plane_id],
+ crtc_state->wm.vlv.raw[VLV_WM_LEVEL_DDR_DVFS].plane[plane_id]);
- wm_state->cxsr = crtc->pipe != PIPE_C && crtc->wm.cxsr_allowed;
- wm_state->num_levels = dev_priv->wm.max_level + 1;
+ return dirty;
+}
- wm_state->num_active_planes = 0;
+static bool vlv_plane_wm_is_valid(const struct intel_crtc_state *crtc_state,
+ enum plane_id plane_id, int level)
+{
+ const struct vlv_pipe_wm *raw =
+ &crtc_state->wm.vlv.raw[level];
+ const struct vlv_fifo_state *fifo_state =
+ &crtc_state->wm.vlv.fifo_state;
- vlv_compute_fifo(crtc);
+ return raw->plane[plane_id] <= fifo_state->plane[plane_id];
+}
- if (wm_state->num_active_planes != 1)
- wm_state->cxsr = false;
+static bool vlv_crtc_wm_is_valid(const struct intel_crtc_state *crtc_state, int level)
+{
+ return vlv_plane_wm_is_valid(crtc_state, PLANE_PRIMARY, level) &&
+ vlv_plane_wm_is_valid(crtc_state, PLANE_SPRITE0, level) &&
+ vlv_plane_wm_is_valid(crtc_state, PLANE_SPRITE1, level) &&
+ vlv_plane_wm_is_valid(crtc_state, PLANE_CURSOR, level);
+}
- for_each_intel_plane_on_crtc(&dev_priv->drm, crtc, plane) {
- struct intel_plane_state *state =
+static int vlv_compute_pipe_wm(struct intel_crtc_state *crtc_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct intel_atomic_state *state =
+ to_intel_atomic_state(crtc_state->base.state);
+ struct vlv_wm_state *wm_state = &crtc_state->wm.vlv.optimal;
+ const struct vlv_fifo_state *fifo_state =
+ &crtc_state->wm.vlv.fifo_state;
+ int num_active_planes = hweight32(crtc_state->active_planes &
+ ~BIT(PLANE_CURSOR));
+ bool needs_modeset = drm_atomic_crtc_needs_modeset(&crtc_state->base);
+ struct intel_plane_state *plane_state;
+ struct intel_plane *plane;
+ enum plane_id plane_id;
+ int level, ret, i;
+ unsigned int dirty = 0;
+
+ for_each_intel_plane_in_state(state, plane, plane_state, i) {
+ const struct intel_plane_state *old_plane_state =
to_intel_plane_state(plane->base.state);
- int level;
- if (!state->base.visible)
+ if (plane_state->base.crtc != &crtc->base &&
+ old_plane_state->base.crtc != &crtc->base)
continue;
- /* normal watermarks */
- for (level = 0; level < wm_state->num_levels; level++) {
- int wm = vlv_compute_wm_level(crtc->config, state, level);
- int max_wm = plane->wm.fifo_size;
+ if (vlv_plane_wm_compute(crtc_state, plane_state))
+ dirty |= BIT(plane->id);
+ }
- /* hack */
- if (WARN_ON(level == 0 && wm > max_wm))
- wm = max_wm;
+ /*
+ * DSPARB registers may have been reset due to the
+ * power well being turned off. Make sure we restore
+ * them to a consistent state even if no primary/sprite
+ * planes are initially active.
+ */
+ if (needs_modeset)
+ crtc_state->fifo_changed = true;
- if (wm > max_wm)
- break;
+ if (!dirty)
+ return 0;
- wm_state->wm[level].plane[plane->id] = wm;
- }
+ /* cursor changes don't warrant a FIFO recompute */
+ if (dirty & ~BIT(PLANE_CURSOR)) {
+ const struct intel_crtc_state *old_crtc_state =
+ to_intel_crtc_state(crtc->base.state);
+ const struct vlv_fifo_state *old_fifo_state =
+ &old_crtc_state->wm.vlv.fifo_state;
- wm_state->num_levels = level;
+ ret = vlv_compute_fifo(crtc_state);
+ if (ret)
+ return ret;
- if (!wm_state->cxsr)
- continue;
+ if (needs_modeset ||
+ memcmp(old_fifo_state, fifo_state,
+ sizeof(*fifo_state)) != 0)
+ crtc_state->fifo_changed = true;
+ }
- /* maxfifo watermarks */
- if (plane->id == PLANE_CURSOR) {
- for (level = 0; level < wm_state->num_levels; level++)
- wm_state->sr[level].cursor =
- wm_state->wm[level].plane[PLANE_CURSOR];
- } else {
- for (level = 0; level < wm_state->num_levels; level++)
- wm_state->sr[level].plane =
- max(wm_state->sr[level].plane,
- wm_state->wm[level].plane[plane->id]);
+ /* initially allow all levels */
+ wm_state->num_levels = vlv_num_wm_levels(dev_priv);
+ /*
+ * Note that enabling cxsr with no primary/sprite planes
+ * enabled can wedge the pipe. Hence we only allow cxsr
+ * with exactly one enabled primary/sprite plane.
+ */
+ wm_state->cxsr = crtc->pipe != PIPE_C && num_active_planes == 1;
+
+ for (level = 0; level < wm_state->num_levels; level++) {
+ const struct vlv_pipe_wm *raw = &crtc_state->wm.vlv.raw[level];
+ const int sr_fifo_size = INTEL_INFO(dev_priv)->num_pipes * 512 - 1;
+
+ if (!vlv_crtc_wm_is_valid(crtc_state, level))
+ break;
+
+ for_each_plane_id_on_crtc(crtc, plane_id) {
+ wm_state->wm[level].plane[plane_id] =
+ vlv_invert_wm_value(raw->plane[plane_id],
+ fifo_state->plane[plane_id]);
}
- }
- /* clear any (partially) filled invalid levels */
- for (level = wm_state->num_levels; level < dev_priv->wm.max_level + 1; level++) {
- memset(&wm_state->wm[level], 0, sizeof(wm_state->wm[level]));
- memset(&wm_state->sr[level], 0, sizeof(wm_state->sr[level]));
+ wm_state->sr[level].plane =
+ vlv_invert_wm_value(max3(raw->plane[PLANE_PRIMARY],
+ raw->plane[PLANE_SPRITE0],
+ raw->plane[PLANE_SPRITE1]),
+ sr_fifo_size);
+
+ wm_state->sr[level].cursor =
+ vlv_invert_wm_value(raw->plane[PLANE_CURSOR],
+ 63);
}
- vlv_invert_wms(crtc);
+ if (level == 0)
+ return -EINVAL;
+
+ /* limit to only levels we can actually handle */
+ wm_state->num_levels = level;
+
+ /* invalidate the higher levels */
+ vlv_invalidate_wms(crtc, wm_state, level);
+
+ return 0;
}
#define VLV_FIFO(plane, value) \
(((value) << DSPARB_ ## plane ## _SHIFT_VLV) & DSPARB_ ## plane ## _MASK_VLV)
-static void vlv_pipe_set_fifo_size(struct intel_crtc *crtc)
+static void vlv_atomic_update_fifo(struct intel_atomic_state *state,
+ struct intel_crtc_state *crtc_state)
{
- struct drm_device *dev = crtc->base.dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_plane *plane;
- int sprite0_start = 0, sprite1_start = 0, fifo_size = 0;
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ const struct vlv_fifo_state *fifo_state =
+ &crtc_state->wm.vlv.fifo_state;
+ int sprite0_start, sprite1_start, fifo_size;
- for_each_intel_plane_on_crtc(dev, crtc, plane) {
- switch (plane->id) {
- case PLANE_PRIMARY:
- sprite0_start = plane->wm.fifo_size;
- break;
- case PLANE_SPRITE0:
- sprite1_start = sprite0_start + plane->wm.fifo_size;
- break;
- case PLANE_SPRITE1:
- fifo_size = sprite1_start + plane->wm.fifo_size;
- break;
- case PLANE_CURSOR:
- WARN_ON(plane->wm.fifo_size != 63);
- break;
- default:
- MISSING_CASE(plane->id);
- break;
- }
- }
+ if (!crtc_state->fifo_changed)
+ return;
- WARN_ON(fifo_size != 512 - 1);
+ sprite0_start = fifo_state->plane[PLANE_PRIMARY];
+ sprite1_start = fifo_state->plane[PLANE_SPRITE0] + sprite0_start;
+ fifo_size = fifo_state->plane[PLANE_SPRITE1] + sprite1_start;
- DRM_DEBUG_KMS("Pipe %c FIFO split %d / %d / %d\n",
- pipe_name(crtc->pipe), sprite0_start,
- sprite1_start, fifo_size);
+ WARN_ON(fifo_state->plane[PLANE_CURSOR] != 63);
+ WARN_ON(fifo_size != 511);
- spin_lock(&dev_priv->wm.dsparb_lock);
+ trace_vlv_fifo_size(crtc, sprite0_start, sprite1_start, fifo_size);
+
+ /*
+ * uncore.lock serves a double purpose here. It allows us to
+ * use the less expensive I915_{READ,WRITE}_FW() functions, and
+ * it protects the DSPARB registers from getting clobbered by
+ * parallel updates from multiple pipes.
+ *
+ * intel_pipe_update_start() has already disabled interrupts
+ * for us, so a plain spin_lock() is sufficient here.
+ */
+ spin_lock(&dev_priv->uncore.lock);
switch (crtc->pipe) {
uint32_t dsparb, dsparb2, dsparb3;
case PIPE_A:
- dsparb = I915_READ(DSPARB);
- dsparb2 = I915_READ(DSPARB2);
+ dsparb = I915_READ_FW(DSPARB);
+ dsparb2 = I915_READ_FW(DSPARB2);
dsparb &= ~(VLV_FIFO(SPRITEA, 0xff) |
VLV_FIFO(SPRITEB, 0xff));
dsparb2 |= (VLV_FIFO(SPRITEA_HI, sprite0_start >> 8) |
VLV_FIFO(SPRITEB_HI, sprite1_start >> 8));
- I915_WRITE(DSPARB, dsparb);
- I915_WRITE(DSPARB2, dsparb2);
+ I915_WRITE_FW(DSPARB, dsparb);
+ I915_WRITE_FW(DSPARB2, dsparb2);
break;
case PIPE_B:
- dsparb = I915_READ(DSPARB);
- dsparb2 = I915_READ(DSPARB2);
+ dsparb = I915_READ_FW(DSPARB);
+ dsparb2 = I915_READ_FW(DSPARB2);
dsparb &= ~(VLV_FIFO(SPRITEC, 0xff) |
VLV_FIFO(SPRITED, 0xff));
dsparb2 |= (VLV_FIFO(SPRITEC_HI, sprite0_start >> 8) |
VLV_FIFO(SPRITED_HI, sprite1_start >> 8));
- I915_WRITE(DSPARB, dsparb);
- I915_WRITE(DSPARB2, dsparb2);
+ I915_WRITE_FW(DSPARB, dsparb);
+ I915_WRITE_FW(DSPARB2, dsparb2);
break;
case PIPE_C:
- dsparb3 = I915_READ(DSPARB3);
- dsparb2 = I915_READ(DSPARB2);
+ dsparb3 = I915_READ_FW(DSPARB3);
+ dsparb2 = I915_READ_FW(DSPARB2);
dsparb3 &= ~(VLV_FIFO(SPRITEE, 0xff) |
VLV_FIFO(SPRITEF, 0xff));
dsparb2 |= (VLV_FIFO(SPRITEE_HI, sprite0_start >> 8) |
VLV_FIFO(SPRITEF_HI, sprite1_start >> 8));
- I915_WRITE(DSPARB3, dsparb3);
- I915_WRITE(DSPARB2, dsparb2);
+ I915_WRITE_FW(DSPARB3, dsparb3);
+ I915_WRITE_FW(DSPARB2, dsparb2);
break;
default:
break;
}
- POSTING_READ(DSPARB);
+ POSTING_READ_FW(DSPARB);
- spin_unlock(&dev_priv->wm.dsparb_lock);
+ spin_unlock(&dev_priv->uncore.lock);
}
#undef VLV_FIFO
+static int vlv_compute_intermediate_wm(struct drm_device *dev,
+ struct intel_crtc *crtc,
+ struct intel_crtc_state *crtc_state)
+{
+ struct vlv_wm_state *intermediate = &crtc_state->wm.vlv.intermediate;
+ const struct vlv_wm_state *optimal = &crtc_state->wm.vlv.optimal;
+ const struct vlv_wm_state *active = &crtc->wm.active.vlv;
+ int level;
+
+ intermediate->num_levels = min(optimal->num_levels, active->num_levels);
+ intermediate->cxsr = optimal->cxsr && active->cxsr &&
+ !crtc_state->disable_cxsr;
+
+ for (level = 0; level < intermediate->num_levels; level++) {
+ enum plane_id plane_id;
+
+ for_each_plane_id_on_crtc(crtc, plane_id) {
+ intermediate->wm[level].plane[plane_id] =
+ min(optimal->wm[level].plane[plane_id],
+ active->wm[level].plane[plane_id]);
+ }
+
+ intermediate->sr[level].plane = min(optimal->sr[level].plane,
+ active->sr[level].plane);
+ intermediate->sr[level].cursor = min(optimal->sr[level].cursor,
+ active->sr[level].cursor);
+ }
+
+ vlv_invalidate_wms(crtc, intermediate, level);
+
+ /*
+ * If our intermediate WM are identical to the final WM, then we can
+ * omit the post-vblank programming; only update if it's different.
+ */
+ if (memcmp(intermediate, optimal, sizeof(*intermediate)) != 0)
+ crtc_state->wm.need_postvbl_update = true;
+
+ return 0;
+}
+
static void vlv_merge_wm(struct drm_i915_private *dev_priv,
struct vlv_wm_values *wm)
{
wm->cxsr = true;
for_each_intel_crtc(&dev_priv->drm, crtc) {
- const struct vlv_wm_state *wm_state = &crtc->wm_state;
+ const struct vlv_wm_state *wm_state = &crtc->wm.active.vlv;
if (!crtc->active)
continue;
wm->level = VLV_WM_LEVEL_PM2;
for_each_intel_crtc(&dev_priv->drm, crtc) {
- struct vlv_wm_state *wm_state = &crtc->wm_state;
+ const struct vlv_wm_state *wm_state = &crtc->wm.active.vlv;
enum pipe pipe = crtc->pipe;
- if (!crtc->active)
- continue;
-
wm->pipe[pipe] = wm_state->wm[wm->level];
- if (wm->cxsr)
+ if (crtc->active && wm->cxsr)
wm->sr = wm_state->sr[wm->level];
wm->ddl[pipe].plane[PLANE_PRIMARY] = DDL_PRECISION_HIGH | 2;
return old < threshold && new >= threshold;
}
-static void vlv_update_wm(struct intel_crtc *crtc)
+static void vlv_program_watermarks(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- enum pipe pipe = crtc->pipe;
struct vlv_wm_values *old_wm = &dev_priv->wm.vlv;
struct vlv_wm_values new_wm = {};
- vlv_compute_wm(crtc);
vlv_merge_wm(dev_priv, &new_wm);
- if (memcmp(old_wm, &new_wm, sizeof(new_wm)) == 0) {
- /* FIXME should be part of crtc atomic commit */
- vlv_pipe_set_fifo_size(crtc);
-
+ if (memcmp(old_wm, &new_wm, sizeof(new_wm)) == 0)
return;
- }
if (is_disabling(old_wm->level, new_wm.level, VLV_WM_LEVEL_DDR_DVFS))
chv_set_memory_dvfs(dev_priv, false);
if (is_disabling(old_wm->cxsr, new_wm.cxsr, true))
_intel_set_memory_cxsr(dev_priv, false);
- /* FIXME should be part of crtc atomic commit */
- vlv_pipe_set_fifo_size(crtc);
-
vlv_write_wm_values(dev_priv, &new_wm);
- DRM_DEBUG_KMS("Setting FIFO watermarks - %c: plane=%d, cursor=%d, "
- "sprite0=%d, sprite1=%d, SR: plane=%d, cursor=%d level=%d cxsr=%d\n",
- pipe_name(pipe), new_wm.pipe[pipe].plane[PLANE_PRIMARY], new_wm.pipe[pipe].plane[PLANE_CURSOR],
- new_wm.pipe[pipe].plane[PLANE_SPRITE0], new_wm.pipe[pipe].plane[PLANE_SPRITE1],
- new_wm.sr.plane, new_wm.sr.cursor, new_wm.level, new_wm.cxsr);
-
if (is_enabling(old_wm->cxsr, new_wm.cxsr, true))
_intel_set_memory_cxsr(dev_priv, true);
*old_wm = new_wm;
}
+static void vlv_initial_watermarks(struct intel_atomic_state *state,
+ struct intel_crtc_state *crtc_state)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+
+ mutex_lock(&dev_priv->wm.wm_mutex);
+ crtc->wm.active.vlv = crtc_state->wm.vlv.intermediate;
+ vlv_program_watermarks(dev_priv);
+ mutex_unlock(&dev_priv->wm.wm_mutex);
+}
+
+static void vlv_optimize_watermarks(struct intel_atomic_state *state,
+ struct intel_crtc_state *crtc_state)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
+ struct intel_crtc *intel_crtc = to_intel_crtc(crtc_state->base.crtc);
+
+ if (!crtc_state->wm.need_postvbl_update)
+ return;
+
+ mutex_lock(&dev_priv->wm.wm_mutex);
+ intel_crtc->wm.active.vlv = crtc_state->wm.vlv.optimal;
+ vlv_program_watermarks(dev_priv);
+ mutex_unlock(&dev_priv->wm.wm_mutex);
+}
+
#define single_plane_enabled(mask) is_power_of_2(mask)
static void g4x_update_wm(struct intel_crtc *crtc)
I915_WRITE(FW_BLC, fwater_lo);
}
-uint32_t ilk_pipe_pixel_rate(const struct intel_crtc_state *pipe_config)
-{
- uint32_t pixel_rate;
-
- pixel_rate = pipe_config->base.adjusted_mode.crtc_clock;
-
- /* We only use IF-ID interlacing. If we ever use PF-ID we'll need to
- * adjust the pixel_rate here. */
-
- if (pipe_config->pch_pfit.enabled) {
- uint64_t pipe_w, pipe_h, pfit_w, pfit_h;
- uint32_t pfit_size = pipe_config->pch_pfit.size;
-
- pipe_w = pipe_config->pipe_src_w;
- pipe_h = pipe_config->pipe_src_h;
-
- pfit_w = (pfit_size >> 16) & 0xFFFF;
- pfit_h = pfit_size & 0xFFFF;
- if (pipe_w < pfit_w)
- pipe_w = pfit_w;
- if (pipe_h < pfit_h)
- pipe_h = pfit_h;
-
- if (WARN_ON(!pfit_w || !pfit_h))
- return pixel_rate;
-
- pixel_rate = div_u64((uint64_t) pixel_rate * pipe_w * pipe_h,
- pfit_w * pfit_h);
- }
-
- return pixel_rate;
-}
-
/* latency must be in 0.1us units. */
static uint32_t ilk_wm_method1(uint32_t pixel_rate, uint8_t cpp, uint32_t latency)
{
cpp = pstate->base.fb->format->cpp[0];
- method1 = ilk_wm_method1(ilk_pipe_pixel_rate(cstate), cpp, mem_value);
+ method1 = ilk_wm_method1(cstate->pixel_rate, cpp, mem_value);
if (!is_lp)
return method1;
- method2 = ilk_wm_method2(ilk_pipe_pixel_rate(cstate),
+ method2 = ilk_wm_method2(cstate->pixel_rate,
cstate->base.adjusted_mode.crtc_htotal,
drm_rect_width(&pstate->base.dst),
cpp, mem_value);
cpp = pstate->base.fb->format->cpp[0];
- method1 = ilk_wm_method1(ilk_pipe_pixel_rate(cstate), cpp, mem_value);
- method2 = ilk_wm_method2(ilk_pipe_pixel_rate(cstate),
+ method1 = ilk_wm_method1(cstate->pixel_rate, cpp, mem_value);
+ method2 = ilk_wm_method2(cstate->pixel_rate,
cstate->base.adjusted_mode.crtc_htotal,
drm_rect_width(&pstate->base.dst),
cpp, mem_value);
const struct intel_plane_state *pstate,
uint32_t mem_value)
{
+ int cpp;
+
/*
- * We treat the cursor plane as always-on for the purposes of watermark
- * calculation. Until we have two-stage watermark programming merged,
- * this is necessary to avoid flickering.
+ * Treat cursor with fb as always visible since cursor updates
+ * can happen faster than the vrefresh rate, and the current
+ * watermark code doesn't handle that correctly. Cursor updates
+ * which set/clear the fb or change the cursor size are going
+ * to get throttled by intel_legacy_cursor_update() to work
+ * around this problem with the watermark code.
*/
- int cpp = 4;
- int width = pstate->base.visible ? pstate->base.crtc_w : 64;
-
- if (!cstate->base.active)
+ if (!cstate->base.active || !pstate->base.fb)
return 0;
- return ilk_wm_method2(ilk_pipe_pixel_rate(cstate),
+ cpp = pstate->base.fb->format->cpp[0];
+
+ return ilk_wm_method2(cstate->pixel_rate,
cstate->base.adjusted_mode.crtc_htotal,
- width, cpp, mem_value);
+ pstate->base.crtc_w, cpp, mem_value);
}
/* Only for WM_LP. */
return 0;
if (WARN_ON(adjusted_mode->crtc_clock == 0))
return 0;
- if (WARN_ON(intel_state->cdclk == 0))
+ if (WARN_ON(intel_state->cdclk.logical.cdclk == 0))
return 0;
/* The WM are computed with base on how long it takes to fill a single
linetime = DIV_ROUND_CLOSEST(adjusted_mode->crtc_htotal * 1000 * 8,
adjusted_mode->crtc_clock);
ips_linetime = DIV_ROUND_CLOSEST(adjusted_mode->crtc_htotal * 1000 * 8,
- intel_state->cdclk);
+ intel_state->cdclk.logical.cdclk);
return PIPE_WM_LINETIME_IPS_LINETIME(ips_linetime) |
PIPE_WM_LINETIME_TIME(linetime);
}
/*
- * WaWmMemoryReadLatency:skl
+ * WaWmMemoryReadLatency:skl,glk
*
* punit doesn't take into account the read latency so we need
* to add 2us to the various latency levels we retrieve from the
* If our intermediate WM are identical to the final WM, then we can
* omit the post-vblank programming; only update if it's different.
*/
- if (memcmp(a, &newstate->wm.ilk.optimal, sizeof(*a)) == 0)
- newstate->wm.need_postvbl_update = false;
+ if (memcmp(a, &newstate->wm.ilk.optimal, sizeof(*a)) != 0)
+ newstate->wm.need_postvbl_update = true;
return 0;
}
{
struct drm_i915_private *dev_priv = to_i915(state->base.dev);
- if (IS_SKYLAKE(dev_priv) || IS_BROXTON(dev_priv) ||
- IS_KABYLAKE(dev_priv))
+ if (IS_GEN9_BC(dev_priv) || IS_BROXTON(dev_priv))
return true;
return false;
* Adjusted plane pixel rate is just the pipe's adjusted pixel rate
* with additional adjustments for plane-specific scaling.
*/
- adjusted_pixel_rate = ilk_pipe_pixel_rate(cstate);
+ adjusted_pixel_rate = cstate->pixel_rate;
downscale_amount = skl_plane_downscale_amount(pstate);
pixel_rate = adjusted_pixel_rate * downscale_amount >> 16;
if (!cstate->base.active)
return 0;
- pixel_rate = ilk_pipe_pixel_rate(cstate);
+ pixel_rate = cstate->pixel_rate;
if (WARN_ON(pixel_rate == 0))
return 0;
struct drm_crtc_state *cstate;
uint32_t i, ret = 0;
- for_each_crtc_in_state(state, crtc, cstate, i)
+ for_each_new_crtc_in_state(state, crtc, cstate, i)
ret |= drm_crtc_mask(crtc);
return ret;
const struct skl_ddb_allocation *new_ddb = &intel_state->wm_results.ddb;
int i;
- for_each_crtc_in_state(state, crtc, cstate, i) {
+ for_each_new_crtc_in_state(state, crtc, cstate, i) {
const struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
enum pipe pipe = intel_crtc->pipe;
* since any racing commits that want to update them would need to
* hold _all_ CRTC state mutexes.
*/
- for_each_crtc_in_state(state, crtc, cstate, i)
+ for_each_new_crtc_in_state(state, crtc, cstate, i)
changed = true;
if (!changed)
return 0;
* should allow skl_update_pipe_wm() to return failure in cases where
* no suitable watermark values can be found.
*/
- for_each_crtc_in_state(state, crtc, cstate, i) {
+ for_each_new_crtc_in_state(state, crtc, cstate, i) {
struct intel_crtc_state *intel_cstate =
to_intel_crtc_state(cstate);
const struct skl_pipe_wm *old_pipe_wm =
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct vlv_wm_values *wm = &dev_priv->wm.vlv;
- struct intel_plane *plane;
- enum pipe pipe;
+ struct intel_crtc *crtc;
u32 val;
vlv_read_wm_values(dev_priv, wm);
- for_each_intel_plane(dev, plane)
- plane->wm.fifo_size = vlv_get_fifo_size(plane);
-
wm->cxsr = I915_READ(FW_BLC_SELF_VLV) & FW_CSPWRDWNEN;
wm->level = VLV_WM_LEVEL_PM2;
mutex_unlock(&dev_priv->rps.hw_lock);
}
- for_each_pipe(dev_priv, pipe)
+ for_each_intel_crtc(dev, crtc) {
+ struct intel_crtc_state *crtc_state =
+ to_intel_crtc_state(crtc->base.state);
+ struct vlv_wm_state *active = &crtc->wm.active.vlv;
+ const struct vlv_fifo_state *fifo_state =
+ &crtc_state->wm.vlv.fifo_state;
+ enum pipe pipe = crtc->pipe;
+ enum plane_id plane_id;
+ int level;
+
+ vlv_get_fifo_size(crtc_state);
+
+ active->num_levels = wm->level + 1;
+ active->cxsr = wm->cxsr;
+
+ for (level = 0; level < active->num_levels; level++) {
+ struct vlv_pipe_wm *raw =
+ &crtc_state->wm.vlv.raw[level];
+
+ active->sr[level].plane = wm->sr.plane;
+ active->sr[level].cursor = wm->sr.cursor;
+
+ for_each_plane_id_on_crtc(crtc, plane_id) {
+ active->wm[level].plane[plane_id] =
+ wm->pipe[pipe].plane[plane_id];
+
+ raw->plane[plane_id] =
+ vlv_invert_wm_value(active->wm[level].plane[plane_id],
+ fifo_state->plane[plane_id]);
+ }
+ }
+
+ for_each_plane_id_on_crtc(crtc, plane_id)
+ vlv_raw_plane_wm_set(crtc_state, level,
+ plane_id, USHRT_MAX);
+ vlv_invalidate_wms(crtc, active, level);
+
+ crtc_state->wm.vlv.optimal = *active;
+ crtc_state->wm.vlv.intermediate = *active;
+
DRM_DEBUG_KMS("Initial watermarks: pipe %c, plane=%d, cursor=%d, sprite0=%d, sprite1=%d\n",
pipe_name(pipe),
wm->pipe[pipe].plane[PLANE_PRIMARY],
wm->pipe[pipe].plane[PLANE_CURSOR],
wm->pipe[pipe].plane[PLANE_SPRITE0],
wm->pipe[pipe].plane[PLANE_SPRITE1]);
+ }
DRM_DEBUG_KMS("Initial watermarks: SR plane=%d, SR cursor=%d level=%d cxsr=%d\n",
wm->sr.plane, wm->sr.cursor, wm->level, wm->cxsr);
}
+void vlv_wm_sanitize(struct drm_i915_private *dev_priv)
+{
+ struct intel_plane *plane;
+ struct intel_crtc *crtc;
+
+ mutex_lock(&dev_priv->wm.wm_mutex);
+
+ for_each_intel_plane(&dev_priv->drm, plane) {
+ struct intel_crtc *crtc =
+ intel_get_crtc_for_pipe(dev_priv, plane->pipe);
+ struct intel_crtc_state *crtc_state =
+ to_intel_crtc_state(crtc->base.state);
+ struct intel_plane_state *plane_state =
+ to_intel_plane_state(plane->base.state);
+ struct vlv_wm_state *wm_state = &crtc_state->wm.vlv.optimal;
+ const struct vlv_fifo_state *fifo_state =
+ &crtc_state->wm.vlv.fifo_state;
+ enum plane_id plane_id = plane->id;
+ int level;
+
+ if (plane_state->base.visible)
+ continue;
+
+ for (level = 0; level < wm_state->num_levels; level++) {
+ struct vlv_pipe_wm *raw =
+ &crtc_state->wm.vlv.raw[level];
+
+ raw->plane[plane_id] = 0;
+
+ wm_state->wm[level].plane[plane_id] =
+ vlv_invert_wm_value(raw->plane[plane_id],
+ fifo_state->plane[plane_id]);
+ }
+ }
+
+ for_each_intel_crtc(&dev_priv->drm, crtc) {
+ struct intel_crtc_state *crtc_state =
+ to_intel_crtc_state(crtc->base.state);
+
+ crtc_state->wm.vlv.intermediate =
+ crtc_state->wm.vlv.optimal;
+ crtc->wm.active.vlv = crtc_state->wm.vlv.optimal;
+ }
+
+ vlv_program_watermarks(dev_priv);
+
+ mutex_unlock(&dev_priv->wm.wm_mutex);
+}
+
void ilk_wm_get_hw_state(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
{
u16 rgvswctl;
- assert_spin_locked(&mchdev_lock);
+ lockdep_assert_held(&mchdev_lock);
rgvswctl = I915_READ16(MEMSWCTL);
if (rgvswctl & MEMCTL_CMD_STS) {
/* gen6_set_rps is called to update the frequency request, but should also be
* called when the range (min_delay and max_delay) is modified so that we can
* update the GEN6_RP_INTERRUPT_LIMITS register accordingly. */
-static void gen6_set_rps(struct drm_i915_private *dev_priv, u8 val)
+static int gen6_set_rps(struct drm_i915_private *dev_priv, u8 val)
{
- /* WaGsvDisableTurbo: Workaround to disable turbo on BXT A* */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1))
- return;
-
- WARN_ON(!mutex_is_locked(&dev_priv->rps.hw_lock));
- WARN_ON(val > dev_priv->rps.max_freq);
- WARN_ON(val < dev_priv->rps.min_freq);
-
/* min/max delay may still have been modified so be sure to
* write the limits value.
*/
I915_WRITE(GEN6_RP_INTERRUPT_LIMITS, intel_rps_limits(dev_priv, val));
I915_WRITE(GEN6_PMINTRMSK, gen6_rps_pm_mask(dev_priv, val));
- POSTING_READ(GEN6_RPNSWREQ);
-
dev_priv->rps.cur_freq = val;
trace_intel_gpu_freq_change(intel_gpu_freq(dev_priv, val));
+
+ return 0;
}
-static void valleyview_set_rps(struct drm_i915_private *dev_priv, u8 val)
+static int valleyview_set_rps(struct drm_i915_private *dev_priv, u8 val)
{
- WARN_ON(!mutex_is_locked(&dev_priv->rps.hw_lock));
- WARN_ON(val > dev_priv->rps.max_freq);
- WARN_ON(val < dev_priv->rps.min_freq);
+ int err;
if (WARN_ONCE(IS_CHERRYVIEW(dev_priv) && (val & 1),
"Odd GPU freq value\n"))
I915_WRITE(GEN6_PMINTRMSK, gen6_rps_pm_mask(dev_priv, val));
if (val != dev_priv->rps.cur_freq) {
- vlv_punit_write(dev_priv, PUNIT_REG_GPU_FREQ_REQ, val);
- if (!IS_CHERRYVIEW(dev_priv))
- gen6_set_rps_thresholds(dev_priv, val);
+ err = vlv_punit_write(dev_priv, PUNIT_REG_GPU_FREQ_REQ, val);
+ if (err)
+ return err;
+
+ gen6_set_rps_thresholds(dev_priv, val);
}
dev_priv->rps.cur_freq = val;
trace_intel_gpu_freq_change(intel_gpu_freq(dev_priv, val));
+
+ return 0;
}
/* vlv_set_rps_idle: Set the frequency to idle, if Gfx clocks are down
static void vlv_set_rps_idle(struct drm_i915_private *dev_priv)
{
u32 val = dev_priv->rps.idle_freq;
+ int err;
if (dev_priv->rps.cur_freq <= val)
return;
* power than the render powerwell.
*/
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_MEDIA);
- valleyview_set_rps(dev_priv, val);
+ err = valleyview_set_rps(dev_priv, val);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_MEDIA);
+
+ if (err)
+ DRM_ERROR("Failed to set RPS for idle\n");
}
void gen6_rps_busy(struct drm_i915_private *dev_priv)
{
mutex_lock(&dev_priv->rps.hw_lock);
if (dev_priv->rps.enabled) {
+ u8 freq;
+
if (dev_priv->pm_rps_events & GEN6_PM_RP_UP_EI_EXPIRED)
gen6_rps_reset_ei(dev_priv);
I915_WRITE(GEN6_PMINTRMSK,
gen6_enable_rps_interrupts(dev_priv);
- /* Ensure we start at the user's desired frequency */
- intel_set_rps(dev_priv,
- clamp(dev_priv->rps.cur_freq,
- dev_priv->rps.min_freq_softlimit,
- dev_priv->rps.max_freq_softlimit));
+ /* Use the user's desired frequency as a guide, but for better
+ * performance, jump directly to RPe as our starting frequency.
+ */
+ freq = max(dev_priv->rps.cur_freq,
+ dev_priv->rps.efficient_freq);
+
+ if (intel_set_rps(dev_priv,
+ clamp(freq,
+ dev_priv->rps.min_freq_softlimit,
+ dev_priv->rps.max_freq_softlimit)))
+ DRM_DEBUG_DRIVER("Failed to set idle frequency\n");
}
mutex_unlock(&dev_priv->rps.hw_lock);
}
spin_unlock(&dev_priv->rps.client_lock);
}
-void intel_set_rps(struct drm_i915_private *dev_priv, u8 val)
+int intel_set_rps(struct drm_i915_private *dev_priv, u8 val)
{
+ int err;
+
+ lockdep_assert_held(&dev_priv->rps.hw_lock);
+ GEM_BUG_ON(val > dev_priv->rps.max_freq);
+ GEM_BUG_ON(val < dev_priv->rps.min_freq);
+
+ if (!dev_priv->rps.enabled) {
+ dev_priv->rps.cur_freq = val;
+ return 0;
+ }
+
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
- valleyview_set_rps(dev_priv, val);
+ err = valleyview_set_rps(dev_priv, val);
else
- gen6_set_rps(dev_priv, val);
+ err = gen6_set_rps(dev_priv, val);
+
+ return err;
}
static void gen9_disable_rc6(struct drm_i915_private *dev_priv)
dev_priv->rps.efficient_freq = dev_priv->rps.rp1_freq;
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv) ||
- IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ IS_GEN9_BC(dev_priv)) {
u32 ddcc_status = 0;
if (sandybridge_pcode_read(dev_priv,
dev_priv->rps.max_freq);
}
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ if (IS_GEN9_BC(dev_priv)) {
/* Store the frequency values in 16.66 MHZ units, which is
* the natural hardware unit for SKL
*/
}
static void reset_rps(struct drm_i915_private *dev_priv,
- void (*set)(struct drm_i915_private *, u8))
+ int (*set)(struct drm_i915_private *, u8))
{
u8 freq = dev_priv->rps.cur_freq;
dev_priv->rps.power = -1;
dev_priv->rps.cur_freq = -1;
- set(dev_priv, freq);
+ if (set(dev_priv, freq))
+ DRM_ERROR("Failed to reset RPS to initial values\n");
}
/* See the Gen9_GT_PM_Programming_Guide doc for the below */
{
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
- /* WaGsvDisableTurbo: Workaround to disable turbo on BXT A* */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1)) {
- /*
- * BIOS could leave the Hw Turbo enabled, so need to explicitly
- * clear out the Control register just to avoid inconsitency
- * with debugfs interface, which will show Turbo as enabled
- * only and that is not expected by the User after adding the
- * WaGsvDisableTurbo. Apart from this there is no problem even
- * if the Turbo is left enabled in the Control register, as the
- * Up/Down interrupts would remain masked.
- */
- gen9_disable_rps(dev_priv);
- intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
- return;
- }
-
/* Program defaults and thresholds for RPS*/
I915_WRITE(GEN6_RC_VIDEO_FREQ,
GEN9_FREQUENCY(dev_priv->rps.rp1_freq));
if (intel_enable_rc6() & INTEL_RC6_ENABLE)
rc6_mask = GEN6_RC_CTL_RC6_ENABLE;
DRM_INFO("RC6 %s\n", onoff(rc6_mask & GEN6_RC_CTL_RC6_ENABLE));
- /* WaRsUseTimeoutMode:bxt */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1)) {
- I915_WRITE(GEN6_RC6_THRESHOLD, 625); /* 800us */
- I915_WRITE(GEN6_RC_CONTROL, GEN6_RC_CTL_HW_ENABLE |
- GEN7_RC_CTL_TO_MODE |
- rc6_mask);
- } else {
- I915_WRITE(GEN6_RC6_THRESHOLD, 37500); /* 37.5/125ms per EI */
- I915_WRITE(GEN6_RC_CONTROL, GEN6_RC_CTL_HW_ENABLE |
- GEN6_RC_CTL_EI_MODE(1) |
- rc6_mask);
- }
+ I915_WRITE(GEN6_RC6_THRESHOLD, 37500); /* 37.5/125ms per EI */
+ I915_WRITE(GEN6_RC_CONTROL,
+ GEN6_RC_CTL_HW_ENABLE | GEN6_RC_CTL_EI_MODE(1) | rc6_mask);
/*
* 3b: Enable Coarse Power Gating only when RC6 is enabled.
/* convert DDR frequency from units of 266.6MHz to bandwidth */
min_ring_freq = mult_frac(min_ring_freq, 8, 3);
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ if (IS_GEN9_BC(dev_priv)) {
/* Convert GT frequency to 50 HZ units */
min_gpu_freq = dev_priv->rps.min_freq / GEN9_FREQ_SCALER;
max_gpu_freq = dev_priv->rps.max_freq / GEN9_FREQ_SCALER;
int diff = max_gpu_freq - gpu_freq;
unsigned int ia_freq = 0, ring_freq = 0;
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ if (IS_GEN9_BC(dev_priv)) {
/*
* ring_freq = 2 * GT. ring_freq is in 100MHz units
* No floor required for ring frequency on SKL.
return rp1;
}
+static u32 cherryview_rps_min_freq(struct drm_i915_private *dev_priv)
+{
+ u32 val, rpn;
+
+ val = vlv_punit_read(dev_priv, FB_GFX_FMIN_AT_VMIN_FUSE);
+ rpn = ((val >> FB_GFX_FMIN_AT_VMIN_FUSE_SHIFT) &
+ FB_GFX_FREQ_FUSE_MASK);
+
+ return rpn;
+}
+
static int valleyview_rps_guar_freq(struct drm_i915_private *dev_priv)
{
u32 val, rp1;
intel_gpu_freq(dev_priv, dev_priv->rps.rp1_freq),
dev_priv->rps.rp1_freq);
- /* PUnit validated range is only [RPe, RP0] */
- dev_priv->rps.min_freq = dev_priv->rps.efficient_freq;
+ dev_priv->rps.min_freq = cherryview_rps_min_freq(dev_priv);
DRM_DEBUG_DRIVER("min GPU freq: %d MHz (%u)\n",
intel_gpu_freq(dev_priv, dev_priv->rps.min_freq),
dev_priv->rps.min_freq);
/* allows RC6 residency counter to work */
I915_WRITE(VLV_COUNTER_CONTROL,
- _MASKED_BIT_ENABLE(VLV_MEDIA_RC0_COUNT_EN |
+ _MASKED_BIT_ENABLE(VLV_COUNT_RANGE_HIGH |
+ VLV_MEDIA_RC0_COUNT_EN |
VLV_RENDER_RC0_COUNT_EN |
VLV_MEDIA_RC6_COUNT_EN |
VLV_RENDER_RC6_COUNT_EN));
unsigned long now = jiffies_to_msecs(jiffies), diff1;
int i;
- assert_spin_locked(&mchdev_lock);
+ lockdep_assert_held(&mchdev_lock);
diff1 = now - dev_priv->ips.last_time1;
u64 now, diff, diffms;
u32 count;
- assert_spin_locked(&mchdev_lock);
+ lockdep_assert_held(&mchdev_lock);
now = ktime_get_raw_ns();
diffms = now - dev_priv->ips.last_time2;
unsigned long t, corr, state1, corr2, state2;
u32 pxvid, ext_v;
- assert_spin_locked(&mchdev_lock);
+ lockdep_assert_held(&mchdev_lock);
pxvid = I915_READ(PXVFREQ(dev_priv->rps.cur_freq));
pxvid = (pxvid >> 24) & 0x7f;
} else if (INTEL_GEN(dev_priv) >= 9) {
gen9_enable_rc6(dev_priv);
gen9_enable_rps(dev_priv);
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ if (IS_GEN9_BC(dev_priv))
gen6_update_ring_freq(dev_priv);
} else if (IS_BROADWELL(dev_priv)) {
gen8_enable_rps(dev_priv);
rcs->init_context(req);
/* Mark the device busy, calling intel_enable_gt_powersave() */
- i915_add_request_no_flush(req);
+ i915_add_request(req);
unlock:
mutex_unlock(&dev_priv->drm.struct_mutex);
| KVM_CONFIG_CHANGE_NOTIFICATION_SELECT);
lpt_init_clock_gating(dev_priv);
+
+ /* WaDisableDopClockGating:bdw
+ *
+ * Also see the CHICKEN2 write in bdw_init_workarounds() to disable DOP
+ * clock gating.
+ */
+ I915_WRITE(GEN6_UCGCTL1,
+ I915_READ(GEN6_UCGCTL1) | GEN6_EU_TCUNIT_CLOCK_GATE_DISABLE);
}
static void haswell_init_clock_gating(struct drm_i915_private *dev_priv)
dev_priv->display.init_clock_gating = skylake_init_clock_gating;
else if (IS_KABYLAKE(dev_priv))
dev_priv->display.init_clock_gating = kabylake_init_clock_gating;
- else if (IS_GEN9_LP(dev_priv))
+ else if (IS_BROXTON(dev_priv))
dev_priv->display.init_clock_gating = bxt_init_clock_gating;
+ else if (IS_GEMINILAKE(dev_priv))
+ dev_priv->display.init_clock_gating = glk_init_clock_gating;
else if (IS_BROADWELL(dev_priv))
dev_priv->display.init_clock_gating = broadwell_init_clock_gating;
else if (IS_CHERRYVIEW(dev_priv))
}
} else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
vlv_setup_wm_latency(dev_priv);
- dev_priv->display.update_wm = vlv_update_wm;
+ dev_priv->display.compute_pipe_wm = vlv_compute_pipe_wm;
+ dev_priv->display.compute_intermediate_wm = vlv_compute_intermediate_wm;
+ dev_priv->display.initial_watermarks = vlv_initial_watermarks;
+ dev_priv->display.optimize_watermarks = vlv_optimize_watermarks;
+ dev_priv->display.atomic_update_watermarks = vlv_atomic_update_fifo;
} else if (IS_PINEVIEW(dev_priv)) {
if (!intel_get_cxsr_latency(IS_PINEVIEW_G(dev_priv),
dev_priv->is_ddr3,
dev_priv->pm.suspended = false;
atomic_set(&dev_priv->pm.wakeref_count, 0);
}
+
+static u64 vlv_residency_raw(struct drm_i915_private *dev_priv,
+ const i915_reg_t reg)
+{
+ u32 lower, upper, tmp;
+
+ /* The register accessed do not need forcewake. We borrow
+ * uncore lock to prevent concurrent access to range reg.
+ */
+ spin_lock_irq(&dev_priv->uncore.lock);
+
+ /* vlv and chv residency counters are 40 bits in width.
+ * With a control bit, we can choose between upper or lower
+ * 32bit window into this counter.
+ *
+ * Although we always use the counter in high-range mode elsewhere,
+ * userspace may attempt to read the value before rc6 is initialised,
+ * before we have set the default VLV_COUNTER_CONTROL value. So always
+ * set the high bit to be safe.
+ */
+ I915_WRITE_FW(VLV_COUNTER_CONTROL,
+ _MASKED_BIT_ENABLE(VLV_COUNT_RANGE_HIGH));
+ upper = I915_READ_FW(reg);
+ do {
+ tmp = upper;
+
+ I915_WRITE_FW(VLV_COUNTER_CONTROL,
+ _MASKED_BIT_DISABLE(VLV_COUNT_RANGE_HIGH));
+ lower = I915_READ_FW(reg);
+
+ I915_WRITE_FW(VLV_COUNTER_CONTROL,
+ _MASKED_BIT_ENABLE(VLV_COUNT_RANGE_HIGH));
+ upper = I915_READ_FW(reg);
+ } while (upper != tmp);
+
+ /* Everywhere else we always use VLV_COUNTER_CONTROL with the
+ * VLV_COUNT_RANGE_HIGH bit set - so it is safe to leave it set
+ * now.
+ */
+
+ spin_unlock_irq(&dev_priv->uncore.lock);
+
+ return lower | (u64)upper << 8;
+}
+
+u64 intel_rc6_residency_us(struct drm_i915_private *dev_priv,
+ const i915_reg_t reg)
+{
+ u64 time_hw, units, div;
+
+ if (!intel_enable_rc6())
+ return 0;
+
+ intel_runtime_pm_get(dev_priv);
+
+ /* On VLV and CHV, residency time is in CZ units rather than 1.28us */
+ if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
+ units = 1000;
+ div = dev_priv->czclk_freq;
+
+ time_hw = vlv_residency_raw(dev_priv, reg);
+ } else if (IS_GEN9_LP(dev_priv)) {
+ units = 1000;
+ div = 1200; /* 833.33ns */
+
+ time_hw = I915_READ(reg);
+ } else {
+ units = 128000; /* 1.28us */
+ div = 100000;
+
+ time_hw = I915_READ(reg);
+ }
+
+ intel_runtime_pm_put(dev_priv);
+ return DIV_ROUND_UP_ULL(time_hw * units, div);
+}
*/
#define LEGACY_REQUEST_SIZE 200
-int __intel_ring_space(int head, int tail, int size)
+static int __intel_ring_space(int head, int tail, int size)
{
int space = head - tail;
if (space <= 0)
static int
gen2_render_ring_flush(struct drm_i915_gem_request *req, u32 mode)
{
- struct intel_ring *ring = req->ring;
- u32 cmd;
- int ret;
+ u32 cmd, *cs;
cmd = MI_FLUSH;
if (mode & EMIT_INVALIDATE)
cmd |= MI_READ_FLUSH;
- ret = intel_ring_begin(req, 2);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 2);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, cmd);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = cmd;
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
static int
gen4_render_ring_flush(struct drm_i915_gem_request *req, u32 mode)
{
- struct intel_ring *ring = req->ring;
- u32 cmd;
- int ret;
+ u32 cmd, *cs;
/*
* read/write caches:
cmd |= MI_INVALIDATE_ISP;
}
- ret = intel_ring_begin(req, 2);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 2);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, cmd);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = cmd;
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
static int
intel_emit_post_sync_nonzero_flush(struct drm_i915_gem_request *req)
{
- struct intel_ring *ring = req->ring;
u32 scratch_addr =
i915_ggtt_offset(req->engine->scratch) + 2 * CACHELINE_BYTES;
- int ret;
-
- ret = intel_ring_begin(req, 6);
- if (ret)
- return ret;
-
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(5));
- intel_ring_emit(ring, PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_STALL_AT_SCOREBOARD);
- intel_ring_emit(ring, scratch_addr | PIPE_CONTROL_GLOBAL_GTT);
- intel_ring_emit(ring, 0); /* low dword */
- intel_ring_emit(ring, 0); /* high dword */
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
-
- ret = intel_ring_begin(req, 6);
- if (ret)
- return ret;
-
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(5));
- intel_ring_emit(ring, PIPE_CONTROL_QW_WRITE);
- intel_ring_emit(ring, scratch_addr | PIPE_CONTROL_GLOBAL_GTT);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ u32 *cs;
+
+ cs = intel_ring_begin(req, 6);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
+
+ *cs++ = GFX_OP_PIPE_CONTROL(5);
+ *cs++ = PIPE_CONTROL_CS_STALL | PIPE_CONTROL_STALL_AT_SCOREBOARD;
+ *cs++ = scratch_addr | PIPE_CONTROL_GLOBAL_GTT;
+ *cs++ = 0; /* low dword */
+ *cs++ = 0; /* high dword */
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
+
+ cs = intel_ring_begin(req, 6);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
+
+ *cs++ = GFX_OP_PIPE_CONTROL(5);
+ *cs++ = PIPE_CONTROL_QW_WRITE;
+ *cs++ = scratch_addr | PIPE_CONTROL_GLOBAL_GTT;
+ *cs++ = 0;
+ *cs++ = 0;
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
static int
gen6_render_ring_flush(struct drm_i915_gem_request *req, u32 mode)
{
- struct intel_ring *ring = req->ring;
u32 scratch_addr =
i915_ggtt_offset(req->engine->scratch) + 2 * CACHELINE_BYTES;
- u32 flags = 0;
+ u32 *cs, flags = 0;
int ret;
/* Force SNB workarounds for PIPE_CONTROL flushes */
flags |= PIPE_CONTROL_QW_WRITE | PIPE_CONTROL_CS_STALL;
}
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(4));
- intel_ring_emit(ring, flags);
- intel_ring_emit(ring, scratch_addr | PIPE_CONTROL_GLOBAL_GTT);
- intel_ring_emit(ring, 0);
- intel_ring_advance(ring);
+ *cs++ = GFX_OP_PIPE_CONTROL(4);
+ *cs++ = flags;
+ *cs++ = scratch_addr | PIPE_CONTROL_GLOBAL_GTT;
+ *cs++ = 0;
+ intel_ring_advance(req, cs);
return 0;
}
static int
gen7_render_ring_cs_stall_wa(struct drm_i915_gem_request *req)
{
- struct intel_ring *ring = req->ring;
- int ret;
+ u32 *cs;
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(4));
- intel_ring_emit(ring,
- PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_STALL_AT_SCOREBOARD);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_advance(ring);
+ *cs++ = GFX_OP_PIPE_CONTROL(4);
+ *cs++ = PIPE_CONTROL_CS_STALL | PIPE_CONTROL_STALL_AT_SCOREBOARD;
+ *cs++ = 0;
+ *cs++ = 0;
+ intel_ring_advance(req, cs);
return 0;
}
static int
gen7_render_ring_flush(struct drm_i915_gem_request *req, u32 mode)
{
- struct intel_ring *ring = req->ring;
u32 scratch_addr =
i915_ggtt_offset(req->engine->scratch) + 2 * CACHELINE_BYTES;
- u32 flags = 0;
- int ret;
+ u32 *cs, flags = 0;
/*
* Ensure that any following seqno writes only happen when the render
gen7_render_ring_cs_stall_wa(req);
}
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(4));
- intel_ring_emit(ring, flags);
- intel_ring_emit(ring, scratch_addr);
- intel_ring_emit(ring, 0);
- intel_ring_advance(ring);
-
- return 0;
-}
-
-static int
-gen8_emit_pipe_control(struct drm_i915_gem_request *req,
- u32 flags, u32 scratch_addr)
-{
- struct intel_ring *ring = req->ring;
- int ret;
-
- ret = intel_ring_begin(req, 6);
- if (ret)
- return ret;
-
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
- intel_ring_emit(ring, flags);
- intel_ring_emit(ring, scratch_addr);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_advance(ring);
+ *cs++ = GFX_OP_PIPE_CONTROL(4);
+ *cs++ = flags;
+ *cs++ = scratch_addr;
+ *cs++ = 0;
+ intel_ring_advance(req, cs);
return 0;
}
static int
gen8_render_ring_flush(struct drm_i915_gem_request *req, u32 mode)
{
- u32 scratch_addr =
- i915_ggtt_offset(req->engine->scratch) + 2 * CACHELINE_BYTES;
- u32 flags = 0;
- int ret;
+ u32 flags;
+ u32 *cs;
- flags |= PIPE_CONTROL_CS_STALL;
+ cs = intel_ring_begin(req, mode & EMIT_INVALIDATE ? 12 : 6);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
+
+ flags = PIPE_CONTROL_CS_STALL;
if (mode & EMIT_FLUSH) {
flags |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH;
flags |= PIPE_CONTROL_GLOBAL_GTT_IVB;
/* WaCsStallBeforeStateCacheInvalidate:bdw,chv */
- ret = gen8_emit_pipe_control(req,
- PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_STALL_AT_SCOREBOARD,
- 0);
- if (ret)
- return ret;
+ cs = gen8_emit_pipe_control(cs,
+ PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_STALL_AT_SCOREBOARD,
+ 0);
}
- return gen8_emit_pipe_control(req, flags, scratch_addr);
+ cs = gen8_emit_pipe_control(cs, flags,
+ i915_ggtt_offset(req->engine->scratch) +
+ 2 * CACHELINE_BYTES);
+
+ intel_ring_advance(req, cs);
+
+ return 0;
}
static void ring_setup_phys_status_page(struct intel_engine_cs *engine)
}
}
-static int intel_ring_workarounds_emit(struct drm_i915_gem_request *req)
-{
- struct intel_ring *ring = req->ring;
- struct i915_workarounds *w = &req->i915->workarounds;
- int ret, i;
-
- if (w->count == 0)
- return 0;
-
- ret = req->engine->emit_flush(req, EMIT_BARRIER);
- if (ret)
- return ret;
-
- ret = intel_ring_begin(req, (w->count * 2 + 2));
- if (ret)
- return ret;
-
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(w->count));
- for (i = 0; i < w->count; i++) {
- intel_ring_emit_reg(ring, w->reg[i].addr);
- intel_ring_emit(ring, w->reg[i].value);
- }
- intel_ring_emit(ring, MI_NOOP);
-
- intel_ring_advance(ring);
-
- ret = req->engine->emit_flush(req, EMIT_BARRIER);
- if (ret)
- return ret;
-
- DRM_DEBUG_DRIVER("Number of Workarounds emitted: %d\n", w->count);
-
- return 0;
-}
-
static int intel_rcs_ctx_init(struct drm_i915_gem_request *req)
{
int ret;
return 0;
}
-static int wa_add(struct drm_i915_private *dev_priv,
- i915_reg_t addr,
- const u32 mask, const u32 val)
-{
- const u32 idx = dev_priv->workarounds.count;
-
- if (WARN_ON(idx >= I915_MAX_WA_REGS))
- return -ENOSPC;
-
- dev_priv->workarounds.reg[idx].addr = addr;
- dev_priv->workarounds.reg[idx].value = val;
- dev_priv->workarounds.reg[idx].mask = mask;
-
- dev_priv->workarounds.count++;
-
- return 0;
-}
-
-#define WA_REG(addr, mask, val) do { \
- const int r = wa_add(dev_priv, (addr), (mask), (val)); \
- if (r) \
- return r; \
- } while (0)
-
-#define WA_SET_BIT_MASKED(addr, mask) \
- WA_REG(addr, (mask), _MASKED_BIT_ENABLE(mask))
-
-#define WA_CLR_BIT_MASKED(addr, mask) \
- WA_REG(addr, (mask), _MASKED_BIT_DISABLE(mask))
-
-#define WA_SET_FIELD_MASKED(addr, mask, value) \
- WA_REG(addr, mask, _MASKED_FIELD(mask, value))
-
-#define WA_SET_BIT(addr, mask) WA_REG(addr, mask, I915_READ(addr) | (mask))
-#define WA_CLR_BIT(addr, mask) WA_REG(addr, mask, I915_READ(addr) & ~(mask))
-
-#define WA_WRITE(addr, val) WA_REG(addr, 0xffffffff, val)
-
-static int wa_ring_whitelist_reg(struct intel_engine_cs *engine,
- i915_reg_t reg)
-{
- struct drm_i915_private *dev_priv = engine->i915;
- struct i915_workarounds *wa = &dev_priv->workarounds;
- const uint32_t index = wa->hw_whitelist_count[engine->id];
-
- if (WARN_ON(index >= RING_MAX_NONPRIV_SLOTS))
- return -EINVAL;
-
- WA_WRITE(RING_FORCE_TO_NONPRIV(engine->mmio_base, index),
- i915_mmio_reg_offset(reg));
- wa->hw_whitelist_count[engine->id]++;
-
- return 0;
-}
-
-static int gen8_init_workarounds(struct intel_engine_cs *engine)
-{
- struct drm_i915_private *dev_priv = engine->i915;
-
- WA_SET_BIT_MASKED(INSTPM, INSTPM_FORCE_ORDERING);
-
- /* WaDisableAsyncFlipPerfMode:bdw,chv */
- WA_SET_BIT_MASKED(MI_MODE, ASYNC_FLIP_PERF_DISABLE);
-
- /* WaDisablePartialInstShootdown:bdw,chv */
- WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
- PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
-
- /* Use Force Non-Coherent whenever executing a 3D context. This is a
- * workaround for for a possible hang in the unlikely event a TLB
- * invalidation occurs during a PSD flush.
- */
- /* WaForceEnableNonCoherent:bdw,chv */
- /* WaHdcDisableFetchWhenMasked:bdw,chv */
- WA_SET_BIT_MASKED(HDC_CHICKEN0,
- HDC_DONOT_FETCH_MEM_WHEN_MASKED |
- HDC_FORCE_NON_COHERENT);
-
- /* From the Haswell PRM, Command Reference: Registers, CACHE_MODE_0:
- * "The Hierarchical Z RAW Stall Optimization allows non-overlapping
- * polygons in the same 8x4 pixel/sample area to be processed without
- * stalling waiting for the earlier ones to write to Hierarchical Z
- * buffer."
- *
- * This optimization is off by default for BDW and CHV; turn it on.
- */
- WA_CLR_BIT_MASKED(CACHE_MODE_0_GEN7, HIZ_RAW_STALL_OPT_DISABLE);
-
- /* Wa4x4STCOptimizationDisable:bdw,chv */
- WA_SET_BIT_MASKED(CACHE_MODE_1, GEN8_4x4_STC_OPTIMIZATION_DISABLE);
-
- /*
- * BSpec recommends 8x4 when MSAA is used,
- * however in practice 16x4 seems fastest.
- *
- * Note that PS/WM thread counts depend on the WIZ hashing
- * disable bit, which we don't touch here, but it's good
- * to keep in mind (see 3DSTATE_PS and 3DSTATE_WM).
- */
- WA_SET_FIELD_MASKED(GEN7_GT_MODE,
- GEN6_WIZ_HASHING_MASK,
- GEN6_WIZ_HASHING_16x4);
-
- return 0;
-}
-
-static int bdw_init_workarounds(struct intel_engine_cs *engine)
-{
- struct drm_i915_private *dev_priv = engine->i915;
- int ret;
-
- ret = gen8_init_workarounds(engine);
- if (ret)
- return ret;
-
- /* WaDisableThreadStallDopClockGating:bdw (pre-production) */
- WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
-
- /* WaDisableDopClockGating:bdw */
- WA_SET_BIT_MASKED(GEN7_ROW_CHICKEN2,
- DOP_CLOCK_GATING_DISABLE);
-
- WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
- GEN8_SAMPLER_POWER_BYPASS_DIS);
-
- WA_SET_BIT_MASKED(HDC_CHICKEN0,
- /* WaForceContextSaveRestoreNonCoherent:bdw */
- HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT |
- /* WaDisableFenceDestinationToSLM:bdw (pre-prod) */
- (IS_BDW_GT3(dev_priv) ? HDC_FENCE_DEST_SLM_DISABLE : 0));
-
- return 0;
-}
-
-static int chv_init_workarounds(struct intel_engine_cs *engine)
-{
- struct drm_i915_private *dev_priv = engine->i915;
- int ret;
-
- ret = gen8_init_workarounds(engine);
- if (ret)
- return ret;
-
- /* WaDisableThreadStallDopClockGating:chv */
- WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
-
- /* Improve HiZ throughput on CHV. */
- WA_SET_BIT_MASKED(HIZ_CHICKEN, CHV_HZ_8X8_MODE_IN_1X);
-
- return 0;
-}
-
-static int gen9_init_workarounds(struct intel_engine_cs *engine)
-{
- struct drm_i915_private *dev_priv = engine->i915;
- int ret;
-
- /* WaConextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl */
- I915_WRITE(GEN9_CSFE_CHICKEN1_RCS, _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));
-
- /* WaEnableLbsSlaRetryTimerDecrement:skl,bxt,kbl */
- I915_WRITE(BDW_SCRATCH1, I915_READ(BDW_SCRATCH1) |
- GEN9_LBS_SLA_RETRY_TIMER_DECREMENT_ENABLE);
-
- /* WaDisableKillLogic:bxt,skl,kbl */
- I915_WRITE(GAM_ECOCHK, I915_READ(GAM_ECOCHK) |
- ECOCHK_DIS_TLB);
-
- /* WaClearFlowControlGpgpuContextSave:skl,bxt,kbl */
- /* WaDisablePartialInstShootdown:skl,bxt,kbl */
- WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
- FLOW_CONTROL_ENABLE |
- PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
-
- /* Syncing dependencies between camera and graphics:skl,bxt,kbl */
- WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
- GEN9_DISABLE_OCL_OOB_SUPPRESS_LOGIC);
-
- /* WaDisableDgMirrorFixInHalfSliceChicken5:bxt */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1))
- WA_CLR_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN5,
- GEN9_DG_MIRROR_FIX_ENABLE);
-
- /* WaSetDisablePixMaskCammingAndRhwoInCommonSliceChicken:bxt */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1)) {
- WA_SET_BIT_MASKED(GEN7_COMMON_SLICE_CHICKEN1,
- GEN9_RHWO_OPTIMIZATION_DISABLE);
- /*
- * WA also requires GEN9_SLICE_COMMON_ECO_CHICKEN0[14:14] to be set
- * but we do that in per ctx batchbuffer as there is an issue
- * with this register not getting restored on ctx restore
- */
- }
-
- /* WaEnableSamplerGPGPUPreemptionSupport:skl,bxt,kbl */
- WA_SET_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN7,
- GEN9_ENABLE_GPGPU_PREEMPTION);
-
- /* Wa4x4STCOptimizationDisable:skl,bxt,kbl */
- /* WaDisablePartialResolveInVc:skl,bxt,kbl */
- WA_SET_BIT_MASKED(CACHE_MODE_1, (GEN8_4x4_STC_OPTIMIZATION_DISABLE |
- GEN9_PARTIAL_RESOLVE_IN_VC_DISABLE));
-
- /* WaCcsTlbPrefetchDisable:skl,bxt,kbl */
- WA_CLR_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN5,
- GEN9_CCS_TLB_PREFETCH_ENABLE);
-
- /* WaDisableMaskBasedCammingInRCC:bxt */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1))
- WA_SET_BIT_MASKED(SLICE_ECO_CHICKEN0,
- PIXEL_MASK_CAMMING_DISABLE);
-
- /* WaForceContextSaveRestoreNonCoherent:skl,bxt,kbl */
- WA_SET_BIT_MASKED(HDC_CHICKEN0,
- HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT |
- HDC_FORCE_CSR_NON_COHERENT_OVR_DISABLE);
-
- /* WaForceEnableNonCoherent and WaDisableHDCInvalidation are
- * both tied to WaForceContextSaveRestoreNonCoherent
- * in some hsds for skl. We keep the tie for all gen9. The
- * documentation is a bit hazy and so we want to get common behaviour,
- * even though there is no clear evidence we would need both on kbl/bxt.
- * This area has been source of system hangs so we play it safe
- * and mimic the skl regardless of what bspec says.
- *
- * Use Force Non-Coherent whenever executing a 3D context. This
- * is a workaround for a possible hang in the unlikely event
- * a TLB invalidation occurs during a PSD flush.
- */
-
- /* WaForceEnableNonCoherent:skl,bxt,kbl */
- WA_SET_BIT_MASKED(HDC_CHICKEN0,
- HDC_FORCE_NON_COHERENT);
-
- /* WaDisableHDCInvalidation:skl,bxt,kbl */
- I915_WRITE(GAM_ECOCHK, I915_READ(GAM_ECOCHK) |
- BDW_DISABLE_HDC_INVALIDATION);
-
- /* WaDisableSamplerPowerBypassForSOPingPong:skl,bxt,kbl */
- if (IS_SKYLAKE(dev_priv) ||
- IS_KABYLAKE(dev_priv) ||
- IS_BXT_REVID(dev_priv, 0, BXT_REVID_B0))
- WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
- GEN8_SAMPLER_POWER_BYPASS_DIS);
-
- /* WaDisableSTUnitPowerOptimization:skl,bxt,kbl */
- WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN2, GEN8_ST_PO_DISABLE);
-
- /* WaOCLCoherentLineFlush:skl,bxt,kbl */
- I915_WRITE(GEN8_L3SQCREG4, (I915_READ(GEN8_L3SQCREG4) |
- GEN8_LQSC_FLUSH_COHERENT_LINES));
-
- /* WaVFEStateAfterPipeControlwithMediaStateClear:skl,bxt */
- ret = wa_ring_whitelist_reg(engine, GEN9_CTX_PREEMPT_REG);
- if (ret)
- return ret;
-
- /* WaEnablePreemptionGranularityControlByUMD:skl,bxt,kbl */
- ret= wa_ring_whitelist_reg(engine, GEN8_CS_CHICKEN1);
- if (ret)
- return ret;
-
- /* WaAllowUMDToModifyHDCChicken1:skl,bxt,kbl */
- ret = wa_ring_whitelist_reg(engine, GEN8_HDC_CHICKEN1);
- if (ret)
- return ret;
-
- return 0;
-}
-
-static int skl_tune_iz_hashing(struct intel_engine_cs *engine)
-{
- struct drm_i915_private *dev_priv = engine->i915;
- u8 vals[3] = { 0, 0, 0 };
- unsigned int i;
-
- for (i = 0; i < 3; i++) {
- u8 ss;
-
- /*
- * Only consider slices where one, and only one, subslice has 7
- * EUs
- */
- if (!is_power_of_2(INTEL_INFO(dev_priv)->sseu.subslice_7eu[i]))
- continue;
-
- /*
- * subslice_7eu[i] != 0 (because of the check above) and
- * ss_max == 4 (maximum number of subslices possible per slice)
- *
- * -> 0 <= ss <= 3;
- */
- ss = ffs(INTEL_INFO(dev_priv)->sseu.subslice_7eu[i]) - 1;
- vals[i] = 3 - ss;
- }
-
- if (vals[0] == 0 && vals[1] == 0 && vals[2] == 0)
- return 0;
-
- /* Tune IZ hashing. See intel_device_info_runtime_init() */
- WA_SET_FIELD_MASKED(GEN7_GT_MODE,
- GEN9_IZ_HASHING_MASK(2) |
- GEN9_IZ_HASHING_MASK(1) |
- GEN9_IZ_HASHING_MASK(0),
- GEN9_IZ_HASHING(2, vals[2]) |
- GEN9_IZ_HASHING(1, vals[1]) |
- GEN9_IZ_HASHING(0, vals[0]));
-
- return 0;
-}
-
-static int skl_init_workarounds(struct intel_engine_cs *engine)
-{
- struct drm_i915_private *dev_priv = engine->i915;
- int ret;
-
- ret = gen9_init_workarounds(engine);
- if (ret)
- return ret;
-
- /*
- * Actual WA is to disable percontext preemption granularity control
- * until D0 which is the default case so this is equivalent to
- * !WaDisablePerCtxtPreemptionGranularityControl:skl
- */
- I915_WRITE(GEN7_FF_SLICE_CS_CHICKEN1,
- _MASKED_BIT_ENABLE(GEN9_FFSC_PERCTX_PREEMPT_CTRL));
-
- /* WaEnableGapsTsvCreditFix:skl */
- I915_WRITE(GEN8_GARBCNTL, (I915_READ(GEN8_GARBCNTL) |
- GEN9_GAPS_TSV_CREDIT_DISABLE));
-
- /* WaDisableGafsUnitClkGating:skl */
- WA_SET_BIT(GEN7_UCGCTL4, GEN8_EU_GAUNIT_CLOCK_GATE_DISABLE);
-
- /* WaInPlaceDecompressionHang:skl */
- if (IS_SKL_REVID(dev_priv, SKL_REVID_H0, REVID_FOREVER))
- WA_SET_BIT(GEN9_GAMT_ECO_REG_RW_IA,
- GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
-
- /* WaDisableLSQCROPERFforOCL:skl */
- ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
- if (ret)
- return ret;
-
- return skl_tune_iz_hashing(engine);
-}
-
-static int bxt_init_workarounds(struct intel_engine_cs *engine)
-{
- struct drm_i915_private *dev_priv = engine->i915;
- int ret;
-
- ret = gen9_init_workarounds(engine);
- if (ret)
- return ret;
-
- /* WaStoreMultiplePTEenable:bxt */
- /* This is a requirement according to Hardware specification */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1))
- I915_WRITE(TILECTL, I915_READ(TILECTL) | TILECTL_TLBPF);
-
- /* WaSetClckGatingDisableMedia:bxt */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1)) {
- I915_WRITE(GEN7_MISCCPCTL, (I915_READ(GEN7_MISCCPCTL) &
- ~GEN8_DOP_CLOCK_GATE_MEDIA_ENABLE));
- }
-
- /* WaDisableThreadStallDopClockGating:bxt */
- WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
- STALL_DOP_GATING_DISABLE);
-
- /* WaDisablePooledEuLoadBalancingFix:bxt */
- if (IS_BXT_REVID(dev_priv, BXT_REVID_B0, REVID_FOREVER)) {
- WA_SET_BIT_MASKED(FF_SLICE_CS_CHICKEN2,
- GEN9_POOLED_EU_LOAD_BALANCING_FIX_DISABLE);
- }
-
- /* WaDisableSbeCacheDispatchPortSharing:bxt */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_B0)) {
- WA_SET_BIT_MASKED(
- GEN7_HALF_SLICE_CHICKEN1,
- GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
- }
-
- /* WaDisableObjectLevelPreemptionForTrifanOrPolygon:bxt */
- /* WaDisableObjectLevelPreemptionForInstancedDraw:bxt */
- /* WaDisableObjectLevelPreemtionForInstanceId:bxt */
- /* WaDisableLSQCROPERFforOCL:bxt */
- if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_A1)) {
- ret = wa_ring_whitelist_reg(engine, GEN9_CS_DEBUG_MODE1);
- if (ret)
- return ret;
-
- ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
- if (ret)
- return ret;
- }
-
- /* WaProgramL3SqcReg1DefaultForPerf:bxt */
- if (IS_BXT_REVID(dev_priv, BXT_REVID_B0, REVID_FOREVER))
- I915_WRITE(GEN8_L3SQCREG1, L3_GENERAL_PRIO_CREDITS(62) |
- L3_HIGH_PRIO_CREDITS(2));
-
- /* WaToEnableHwFixForPushConstHWBug:bxt */
- if (IS_BXT_REVID(dev_priv, BXT_REVID_C0, REVID_FOREVER))
- WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
- GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
-
- /* WaInPlaceDecompressionHang:bxt */
- if (IS_BXT_REVID(dev_priv, BXT_REVID_C0, REVID_FOREVER))
- WA_SET_BIT(GEN9_GAMT_ECO_REG_RW_IA,
- GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
-
- return 0;
-}
-
-static int kbl_init_workarounds(struct intel_engine_cs *engine)
-{
- struct drm_i915_private *dev_priv = engine->i915;
- int ret;
-
- ret = gen9_init_workarounds(engine);
- if (ret)
- return ret;
-
- /* WaEnableGapsTsvCreditFix:kbl */
- I915_WRITE(GEN8_GARBCNTL, (I915_READ(GEN8_GARBCNTL) |
- GEN9_GAPS_TSV_CREDIT_DISABLE));
-
- /* WaDisableDynamicCreditSharing:kbl */
- if (IS_KBL_REVID(dev_priv, 0, KBL_REVID_B0))
- WA_SET_BIT(GAMT_CHKN_BIT_REG,
- GAMT_CHKN_DISABLE_DYNAMIC_CREDIT_SHARING);
-
- /* WaDisableFenceDestinationToSLM:kbl (pre-prod) */
- if (IS_KBL_REVID(dev_priv, KBL_REVID_A0, KBL_REVID_A0))
- WA_SET_BIT_MASKED(HDC_CHICKEN0,
- HDC_FENCE_DEST_SLM_DISABLE);
-
- /* WaToEnableHwFixForPushConstHWBug:kbl */
- if (IS_KBL_REVID(dev_priv, KBL_REVID_C0, REVID_FOREVER))
- WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
- GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
-
- /* WaDisableGafsUnitClkGating:kbl */
- WA_SET_BIT(GEN7_UCGCTL4, GEN8_EU_GAUNIT_CLOCK_GATE_DISABLE);
-
- /* WaDisableSbeCacheDispatchPortSharing:kbl */
- WA_SET_BIT_MASKED(
- GEN7_HALF_SLICE_CHICKEN1,
- GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
-
- /* WaInPlaceDecompressionHang:kbl */
- WA_SET_BIT(GEN9_GAMT_ECO_REG_RW_IA,
- GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
-
- /* WaDisableLSQCROPERFforOCL:kbl */
- ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
- if (ret)
- return ret;
-
- return 0;
-}
-
-int init_workarounds_ring(struct intel_engine_cs *engine)
-{
- struct drm_i915_private *dev_priv = engine->i915;
-
- WARN_ON(engine->id != RCS);
-
- dev_priv->workarounds.count = 0;
- dev_priv->workarounds.hw_whitelist_count[RCS] = 0;
-
- if (IS_BROADWELL(dev_priv))
- return bdw_init_workarounds(engine);
-
- if (IS_CHERRYVIEW(dev_priv))
- return chv_init_workarounds(engine);
-
- if (IS_SKYLAKE(dev_priv))
- return skl_init_workarounds(engine);
-
- if (IS_BROXTON(dev_priv))
- return bxt_init_workarounds(engine);
-
- if (IS_KABYLAKE(dev_priv))
- return kbl_init_workarounds(engine);
-
- return 0;
-}
-
static int init_render_ring(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
i915_vma_unpin_and_release(&dev_priv->semaphore);
}
-static u32 *gen8_rcs_signal(struct drm_i915_gem_request *req, u32 *out)
+static u32 *gen8_rcs_signal(struct drm_i915_gem_request *req, u32 *cs)
{
struct drm_i915_private *dev_priv = req->i915;
struct intel_engine_cs *waiter;
if (gtt_offset == MI_SEMAPHORE_SYNC_INVALID)
continue;
- *out++ = GFX_OP_PIPE_CONTROL(6);
- *out++ = (PIPE_CONTROL_GLOBAL_GTT_IVB |
- PIPE_CONTROL_QW_WRITE |
- PIPE_CONTROL_CS_STALL);
- *out++ = lower_32_bits(gtt_offset);
- *out++ = upper_32_bits(gtt_offset);
- *out++ = req->global_seqno;
- *out++ = 0;
- *out++ = (MI_SEMAPHORE_SIGNAL |
- MI_SEMAPHORE_TARGET(waiter->hw_id));
- *out++ = 0;
+ *cs++ = GFX_OP_PIPE_CONTROL(6);
+ *cs++ = PIPE_CONTROL_GLOBAL_GTT_IVB | PIPE_CONTROL_QW_WRITE |
+ PIPE_CONTROL_CS_STALL;
+ *cs++ = lower_32_bits(gtt_offset);
+ *cs++ = upper_32_bits(gtt_offset);
+ *cs++ = req->global_seqno;
+ *cs++ = 0;
+ *cs++ = MI_SEMAPHORE_SIGNAL |
+ MI_SEMAPHORE_TARGET(waiter->hw_id);
+ *cs++ = 0;
}
- return out;
+ return cs;
}
-static u32 *gen8_xcs_signal(struct drm_i915_gem_request *req, u32 *out)
+static u32 *gen8_xcs_signal(struct drm_i915_gem_request *req, u32 *cs)
{
struct drm_i915_private *dev_priv = req->i915;
struct intel_engine_cs *waiter;
if (gtt_offset == MI_SEMAPHORE_SYNC_INVALID)
continue;
- *out++ = (MI_FLUSH_DW + 1) | MI_FLUSH_DW_OP_STOREDW;
- *out++ = lower_32_bits(gtt_offset) | MI_FLUSH_DW_USE_GTT;
- *out++ = upper_32_bits(gtt_offset);
- *out++ = req->global_seqno;
- *out++ = (MI_SEMAPHORE_SIGNAL |
- MI_SEMAPHORE_TARGET(waiter->hw_id));
- *out++ = 0;
+ *cs++ = (MI_FLUSH_DW + 1) | MI_FLUSH_DW_OP_STOREDW;
+ *cs++ = lower_32_bits(gtt_offset) | MI_FLUSH_DW_USE_GTT;
+ *cs++ = upper_32_bits(gtt_offset);
+ *cs++ = req->global_seqno;
+ *cs++ = MI_SEMAPHORE_SIGNAL |
+ MI_SEMAPHORE_TARGET(waiter->hw_id);
+ *cs++ = 0;
}
- return out;
+ return cs;
}
-static u32 *gen6_signal(struct drm_i915_gem_request *req, u32 *out)
+static u32 *gen6_signal(struct drm_i915_gem_request *req, u32 *cs)
{
struct drm_i915_private *dev_priv = req->i915;
struct intel_engine_cs *engine;
mbox_reg = req->engine->semaphore.mbox.signal[engine->hw_id];
if (i915_mmio_reg_valid(mbox_reg)) {
- *out++ = MI_LOAD_REGISTER_IMM(1);
- *out++ = i915_mmio_reg_offset(mbox_reg);
- *out++ = req->global_seqno;
+ *cs++ = MI_LOAD_REGISTER_IMM(1);
+ *cs++ = i915_mmio_reg_offset(mbox_reg);
+ *cs++ = req->global_seqno;
num_rings++;
}
}
if (num_rings & 1)
- *out++ = MI_NOOP;
+ *cs++ = MI_NOOP;
- return out;
+ return cs;
}
static void i9xx_submit_request(struct drm_i915_gem_request *request)
i915_gem_request_submit(request);
+ GEM_BUG_ON(!IS_ALIGNED(request->tail, 8));
I915_WRITE_TAIL(request->engine, request->tail);
}
-static void i9xx_emit_breadcrumb(struct drm_i915_gem_request *req,
- u32 *out)
+static void i9xx_emit_breadcrumb(struct drm_i915_gem_request *req, u32 *cs)
{
- *out++ = MI_STORE_DWORD_INDEX;
- *out++ = I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT;
- *out++ = req->global_seqno;
- *out++ = MI_USER_INTERRUPT;
+ *cs++ = MI_STORE_DWORD_INDEX;
+ *cs++ = I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT;
+ *cs++ = req->global_seqno;
+ *cs++ = MI_USER_INTERRUPT;
- req->tail = intel_ring_offset(req->ring, out);
+ req->tail = intel_ring_offset(req, cs);
+ GEM_BUG_ON(!IS_ALIGNED(req->tail, 8));
}
static const int i9xx_emit_breadcrumb_sz = 4;
* Update the mailbox registers in the *other* rings with the current seqno.
* This acts like a signal in the canonical semaphore.
*/
-static void gen6_sema_emit_breadcrumb(struct drm_i915_gem_request *req,
- u32 *out)
+static void gen6_sema_emit_breadcrumb(struct drm_i915_gem_request *req, u32 *cs)
{
return i9xx_emit_breadcrumb(req,
- req->engine->semaphore.signal(req, out));
+ req->engine->semaphore.signal(req, cs));
}
static void gen8_render_emit_breadcrumb(struct drm_i915_gem_request *req,
- u32 *out)
+ u32 *cs)
{
struct intel_engine_cs *engine = req->engine;
if (engine->semaphore.signal)
- out = engine->semaphore.signal(req, out);
-
- *out++ = GFX_OP_PIPE_CONTROL(6);
- *out++ = (PIPE_CONTROL_GLOBAL_GTT_IVB |
- PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_QW_WRITE);
- *out++ = intel_hws_seqno_address(engine);
- *out++ = 0;
- *out++ = req->global_seqno;
+ cs = engine->semaphore.signal(req, cs);
+
+ *cs++ = GFX_OP_PIPE_CONTROL(6);
+ *cs++ = PIPE_CONTROL_GLOBAL_GTT_IVB | PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_QW_WRITE;
+ *cs++ = intel_hws_seqno_address(engine);
+ *cs++ = 0;
+ *cs++ = req->global_seqno;
/* We're thrashing one dword of HWS. */
- *out++ = 0;
- *out++ = MI_USER_INTERRUPT;
- *out++ = MI_NOOP;
+ *cs++ = 0;
+ *cs++ = MI_USER_INTERRUPT;
+ *cs++ = MI_NOOP;
- req->tail = intel_ring_offset(req->ring, out);
+ req->tail = intel_ring_offset(req, cs);
+ GEM_BUG_ON(!IS_ALIGNED(req->tail, 8));
}
static const int gen8_render_emit_breadcrumb_sz = 8;
gen8_ring_sync_to(struct drm_i915_gem_request *req,
struct drm_i915_gem_request *signal)
{
- struct intel_ring *ring = req->ring;
struct drm_i915_private *dev_priv = req->i915;
u64 offset = GEN8_WAIT_OFFSET(req->engine, signal->engine->id);
struct i915_hw_ppgtt *ppgtt;
- int ret;
+ u32 *cs;
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring,
- MI_SEMAPHORE_WAIT |
- MI_SEMAPHORE_GLOBAL_GTT |
- MI_SEMAPHORE_SAD_GTE_SDD);
- intel_ring_emit(ring, signal->global_seqno);
- intel_ring_emit(ring, lower_32_bits(offset));
- intel_ring_emit(ring, upper_32_bits(offset));
- intel_ring_advance(ring);
+ *cs++ = MI_SEMAPHORE_WAIT | MI_SEMAPHORE_GLOBAL_GTT |
+ MI_SEMAPHORE_SAD_GTE_SDD;
+ *cs++ = signal->global_seqno;
+ *cs++ = lower_32_bits(offset);
+ *cs++ = upper_32_bits(offset);
+ intel_ring_advance(req, cs);
/* When the !RCS engines idle waiting upon a semaphore, they lose their
* pagetables and we must reload them before executing the batch.
gen6_ring_sync_to(struct drm_i915_gem_request *req,
struct drm_i915_gem_request *signal)
{
- struct intel_ring *ring = req->ring;
u32 dw1 = MI_SEMAPHORE_MBOX |
MI_SEMAPHORE_COMPARE |
MI_SEMAPHORE_REGISTER;
u32 wait_mbox = signal->engine->semaphore.mbox.wait[req->engine->hw_id];
- int ret;
+ u32 *cs;
WARN_ON(wait_mbox == MI_SEMAPHORE_SYNC_INVALID);
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, dw1 | wait_mbox);
+ *cs++ = dw1 | wait_mbox;
/* Throughout all of the GEM code, seqno passed implies our current
* seqno is >= the last seqno executed. However for hardware the
* comparison is strictly greater than.
*/
- intel_ring_emit(ring, signal->global_seqno - 1);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = signal->global_seqno - 1;
+ *cs++ = 0;
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
static int
bsd_ring_flush(struct drm_i915_gem_request *req, u32 mode)
{
- struct intel_ring *ring = req->ring;
- int ret;
+ u32 *cs;
- ret = intel_ring_begin(req, 2);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 2);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, MI_FLUSH);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = MI_FLUSH;
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
u64 offset, u32 length,
unsigned int dispatch_flags)
{
- struct intel_ring *ring = req->ring;
- int ret;
+ u32 *cs;
- ret = intel_ring_begin(req, 2);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 2);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring,
- MI_BATCH_BUFFER_START |
- MI_BATCH_GTT |
- (dispatch_flags & I915_DISPATCH_SECURE ?
- 0 : MI_BATCH_NON_SECURE_I965));
- intel_ring_emit(ring, offset);
- intel_ring_advance(ring);
+ *cs++ = MI_BATCH_BUFFER_START | MI_BATCH_GTT | (dispatch_flags &
+ I915_DISPATCH_SECURE ? 0 : MI_BATCH_NON_SECURE_I965);
+ *cs++ = offset;
+ intel_ring_advance(req, cs);
return 0;
}
u64 offset, u32 len,
unsigned int dispatch_flags)
{
- struct intel_ring *ring = req->ring;
- u32 cs_offset = i915_ggtt_offset(req->engine->scratch);
- int ret;
+ u32 *cs, cs_offset = i915_ggtt_offset(req->engine->scratch);
- ret = intel_ring_begin(req, 6);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 6);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
/* Evict the invalid PTE TLBs */
- intel_ring_emit(ring, COLOR_BLT_CMD | BLT_WRITE_RGBA);
- intel_ring_emit(ring, BLT_DEPTH_32 | BLT_ROP_COLOR_COPY | 4096);
- intel_ring_emit(ring, I830_TLB_ENTRIES << 16 | 4); /* load each page */
- intel_ring_emit(ring, cs_offset);
- intel_ring_emit(ring, 0xdeadbeef);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = COLOR_BLT_CMD | BLT_WRITE_RGBA;
+ *cs++ = BLT_DEPTH_32 | BLT_ROP_COLOR_COPY | 4096;
+ *cs++ = I830_TLB_ENTRIES << 16 | 4; /* load each page */
+ *cs++ = cs_offset;
+ *cs++ = 0xdeadbeef;
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
if ((dispatch_flags & I915_DISPATCH_PINNED) == 0) {
if (len > I830_BATCH_LIMIT)
return -ENOSPC;
- ret = intel_ring_begin(req, 6 + 2);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 6 + 2);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
/* Blit the batch (which has now all relocs applied) to the
* stable batch scratch bo area (so that the CS never
* stumbles over its tlb invalidation bug) ...
*/
- intel_ring_emit(ring, SRC_COPY_BLT_CMD | BLT_WRITE_RGBA);
- intel_ring_emit(ring,
- BLT_DEPTH_32 | BLT_ROP_SRC_COPY | 4096);
- intel_ring_emit(ring, DIV_ROUND_UP(len, 4096) << 16 | 4096);
- intel_ring_emit(ring, cs_offset);
- intel_ring_emit(ring, 4096);
- intel_ring_emit(ring, offset);
-
- intel_ring_emit(ring, MI_FLUSH);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = SRC_COPY_BLT_CMD | BLT_WRITE_RGBA;
+ *cs++ = BLT_DEPTH_32 | BLT_ROP_SRC_COPY | 4096;
+ *cs++ = DIV_ROUND_UP(len, 4096) << 16 | 4096;
+ *cs++ = cs_offset;
+ *cs++ = 4096;
+ *cs++ = offset;
+
+ *cs++ = MI_FLUSH;
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
/* ... and execute it. */
offset = cs_offset;
}
- ret = intel_ring_begin(req, 2);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 2);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, MI_BATCH_BUFFER_START | MI_BATCH_GTT);
- intel_ring_emit(ring, offset | (dispatch_flags & I915_DISPATCH_SECURE ?
- 0 : MI_BATCH_NON_SECURE));
- intel_ring_advance(ring);
+ *cs++ = MI_BATCH_BUFFER_START | MI_BATCH_GTT;
+ *cs++ = offset | (dispatch_flags & I915_DISPATCH_SECURE ? 0 :
+ MI_BATCH_NON_SECURE);
+ intel_ring_advance(req, cs);
return 0;
}
u64 offset, u32 len,
unsigned int dispatch_flags)
{
- struct intel_ring *ring = req->ring;
- int ret;
+ u32 *cs;
- ret = intel_ring_begin(req, 2);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 2);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring, MI_BATCH_BUFFER_START | MI_BATCH_GTT);
- intel_ring_emit(ring, offset | (dispatch_flags & I915_DISPATCH_SECURE ?
- 0 : MI_BATCH_NON_SECURE));
- intel_ring_advance(ring);
+ *cs++ = MI_BATCH_BUFFER_START | MI_BATCH_GTT;
+ *cs++ = offset | (dispatch_flags & I915_DISPATCH_SECURE ? 0 :
+ MI_BATCH_NON_SECURE);
+ intel_ring_advance(req, cs);
return 0;
}
kfree(ring);
}
-static int context_pin(struct i915_gem_context *ctx, unsigned int flags)
+static int context_pin(struct i915_gem_context *ctx)
{
struct i915_vma *vma = ctx->engine[RCS].state;
int ret;
return ret;
}
- return i915_vma_pin(vma, 0, ctx->ggtt_alignment, PIN_GLOBAL | flags);
+ return i915_vma_pin(vma, 0, I915_GTT_MIN_ALIGNMENT,
+ PIN_GLOBAL | PIN_HIGH);
}
static int intel_ring_context_pin(struct intel_engine_cs *engine,
if (ce->pin_count++)
return 0;
+ GEM_BUG_ON(!ce->pin_count); /* no overflow please! */
if (ce->state) {
- unsigned int flags;
-
- flags = 0;
- if (i915_gem_context_is_kernel(ctx))
- flags = PIN_HIGH;
-
- ret = context_pin(ctx, flags);
+ ret = context_pin(ctx);
if (ret)
goto error;
}
static int ring_request_alloc(struct drm_i915_gem_request *request)
{
- int ret;
+ u32 *cs;
GEM_BUG_ON(!request->ctx->engine[request->engine->id].pin_count);
GEM_BUG_ON(!request->engine->buffer);
request->ring = request->engine->buffer;
- ret = intel_ring_begin(request, 0);
- if (ret)
- return ret;
+ cs = intel_ring_begin(request, 0);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
request->reserved_space -= LEGACY_REQUEST_SIZE;
return 0;
return 0;
}
-int intel_ring_begin(struct drm_i915_gem_request *req, int num_dwords)
+u32 *intel_ring_begin(struct drm_i915_gem_request *req, int num_dwords)
{
struct intel_ring *ring = req->ring;
int remain_actual = ring->size - ring->tail;
int bytes = num_dwords * sizeof(u32);
int total_bytes, wait_bytes;
bool need_wrap = false;
+ u32 *cs;
total_bytes = bytes + req->reserved_space;
if (wait_bytes > ring->space) {
int ret = wait_for_space(req, wait_bytes);
if (unlikely(ret))
- return ret;
+ return ERR_PTR(ret);
}
if (unlikely(need_wrap)) {
ring->space -= remain_actual;
}
+ GEM_BUG_ON(ring->tail > ring->size - bytes);
+ cs = ring->vaddr + ring->tail;
+ ring->tail += bytes;
ring->space -= bytes;
GEM_BUG_ON(ring->space < 0);
- return 0;
+
+ return cs;
}
/* Align the ring tail to a cacheline boundary */
int intel_ring_cacheline_align(struct drm_i915_gem_request *req)
{
- struct intel_ring *ring = req->ring;
int num_dwords =
- (ring->tail & (CACHELINE_BYTES - 1)) / sizeof(uint32_t);
- int ret;
+ (req->ring->tail & (CACHELINE_BYTES - 1)) / sizeof(uint32_t);
+ u32 *cs;
if (num_dwords == 0)
return 0;
num_dwords = CACHELINE_BYTES / sizeof(uint32_t) - num_dwords;
- ret = intel_ring_begin(req, num_dwords);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, num_dwords);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
while (num_dwords--)
- intel_ring_emit(ring, MI_NOOP);
+ *cs++ = MI_NOOP;
- intel_ring_advance(ring);
+ intel_ring_advance(req, cs);
return 0;
}
static int gen6_bsd_ring_flush(struct drm_i915_gem_request *req, u32 mode)
{
- struct intel_ring *ring = req->ring;
- uint32_t cmd;
- int ret;
+ u32 cmd, *cs;
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
cmd = MI_FLUSH_DW;
if (INTEL_GEN(req->i915) >= 8)
if (mode & EMIT_INVALIDATE)
cmd |= MI_INVALIDATE_TLB | MI_INVALIDATE_BSD;
- intel_ring_emit(ring, cmd);
- intel_ring_emit(ring, I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT);
+ *cs++ = cmd;
+ *cs++ = I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT;
if (INTEL_GEN(req->i915) >= 8) {
- intel_ring_emit(ring, 0); /* upper addr */
- intel_ring_emit(ring, 0); /* value */
+ *cs++ = 0; /* upper addr */
+ *cs++ = 0; /* value */
} else {
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, MI_NOOP);
+ *cs++ = 0;
+ *cs++ = MI_NOOP;
}
- intel_ring_advance(ring);
+ intel_ring_advance(req, cs);
return 0;
}
u64 offset, u32 len,
unsigned int dispatch_flags)
{
- struct intel_ring *ring = req->ring;
bool ppgtt = USES_PPGTT(req->i915) &&
!(dispatch_flags & I915_DISPATCH_SECURE);
- int ret;
+ u32 *cs;
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
/* FIXME(BDW): Address space and security selectors. */
- intel_ring_emit(ring, MI_BATCH_BUFFER_START_GEN8 | (ppgtt<<8) |
- (dispatch_flags & I915_DISPATCH_RS ?
- MI_BATCH_RESOURCE_STREAMER : 0));
- intel_ring_emit(ring, lower_32_bits(offset));
- intel_ring_emit(ring, upper_32_bits(offset));
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ *cs++ = MI_BATCH_BUFFER_START_GEN8 | (ppgtt << 8) | (dispatch_flags &
+ I915_DISPATCH_RS ? MI_BATCH_RESOURCE_STREAMER : 0);
+ *cs++ = lower_32_bits(offset);
+ *cs++ = upper_32_bits(offset);
+ *cs++ = MI_NOOP;
+ intel_ring_advance(req, cs);
return 0;
}
u64 offset, u32 len,
unsigned int dispatch_flags)
{
- struct intel_ring *ring = req->ring;
- int ret;
+ u32 *cs;
- ret = intel_ring_begin(req, 2);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 2);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring,
- MI_BATCH_BUFFER_START |
- (dispatch_flags & I915_DISPATCH_SECURE ?
- 0 : MI_BATCH_PPGTT_HSW | MI_BATCH_NON_SECURE_HSW) |
- (dispatch_flags & I915_DISPATCH_RS ?
- MI_BATCH_RESOURCE_STREAMER : 0));
+ *cs++ = MI_BATCH_BUFFER_START | (dispatch_flags & I915_DISPATCH_SECURE ?
+ 0 : MI_BATCH_PPGTT_HSW | MI_BATCH_NON_SECURE_HSW) |
+ (dispatch_flags & I915_DISPATCH_RS ?
+ MI_BATCH_RESOURCE_STREAMER : 0);
/* bit0-7 is the length on GEN6+ */
- intel_ring_emit(ring, offset);
- intel_ring_advance(ring);
+ *cs++ = offset;
+ intel_ring_advance(req, cs);
return 0;
}
u64 offset, u32 len,
unsigned int dispatch_flags)
{
- struct intel_ring *ring = req->ring;
- int ret;
+ u32 *cs;
- ret = intel_ring_begin(req, 2);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 2);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
- intel_ring_emit(ring,
- MI_BATCH_BUFFER_START |
- (dispatch_flags & I915_DISPATCH_SECURE ?
- 0 : MI_BATCH_NON_SECURE_I965));
+ *cs++ = MI_BATCH_BUFFER_START | (dispatch_flags & I915_DISPATCH_SECURE ?
+ 0 : MI_BATCH_NON_SECURE_I965);
/* bit0-7 is the length on GEN6+ */
- intel_ring_emit(ring, offset);
- intel_ring_advance(ring);
+ *cs++ = offset;
+ intel_ring_advance(req, cs);
return 0;
}
static int gen6_ring_flush(struct drm_i915_gem_request *req, u32 mode)
{
- struct intel_ring *ring = req->ring;
- uint32_t cmd;
- int ret;
+ u32 cmd, *cs;
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
+ cs = intel_ring_begin(req, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
cmd = MI_FLUSH_DW;
if (INTEL_GEN(req->i915) >= 8)
*/
if (mode & EMIT_INVALIDATE)
cmd |= MI_INVALIDATE_TLB;
- intel_ring_emit(ring, cmd);
- intel_ring_emit(ring,
- I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT);
+ *cs++ = cmd;
+ *cs++ = I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT;
if (INTEL_GEN(req->i915) >= 8) {
- intel_ring_emit(ring, 0); /* upper addr */
- intel_ring_emit(ring, 0); /* value */
+ *cs++ = 0; /* upper addr */
+ *cs++ = 0; /* value */
} else {
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, MI_NOOP);
+ *cs++ = 0;
+ *cs++ = MI_NOOP;
}
- intel_ring_advance(ring);
+ intel_ring_advance(req, cs);
return 0;
}
}
}
+static void i9xx_set_default_submission(struct intel_engine_cs *engine)
+{
+ engine->submit_request = i9xx_submit_request;
+}
+
+static void gen6_bsd_set_default_submission(struct intel_engine_cs *engine)
+{
+ engine->submit_request = gen6_bsd_submit_request;
+}
+
static void intel_ring_default_vfuncs(struct drm_i915_private *dev_priv,
struct intel_engine_cs *engine)
{
engine->emit_breadcrumb_sz++;
}
}
- engine->submit_request = i9xx_submit_request;
+
+ engine->set_default_submission = i9xx_set_default_submission;
if (INTEL_GEN(dev_priv) >= 8)
engine->emit_bb_start = gen8_emit_bb_start;
if (INTEL_GEN(dev_priv) >= 6) {
/* gen6 bsd needs a special wa for tail updates */
if (IS_GEN6(dev_priv))
- engine->submit_request = gen6_bsd_submit_request;
+ engine->set_default_submission = gen6_bsd_set_default_submission;
engine->emit_flush = gen6_bsd_ring_flush;
if (INTEL_GEN(dev_priv) < 8)
engine->irq_enable_mask = GT_BSD_USER_INTERRUPT;
#include "i915_gem_batch_pool.h"
#include "i915_gem_request.h"
#include "i915_gem_timeline.h"
+#include "i915_selftest.h"
#define I915_CMD_HASH_ORDER 9
u32 head;
u32 tail;
+
int space;
int size;
int effective_size;
struct drm_i915_gem_request;
struct intel_render_state;
+/*
+ * Engine IDs definitions.
+ * Keep instances of the same type engine together.
+ */
+enum intel_engine_id {
+ RCS = 0,
+ BCS,
+ VCS,
+ VCS2,
+#define _VCS(n) (VCS + (n))
+ VECS
+};
+
struct intel_engine_cs {
struct drm_i915_private *i915;
const char *name;
- enum intel_engine_id {
- RCS = 0,
- BCS,
- VCS,
- VCS2, /* Keep instances of the same type engine together. */
- VECS
- } id;
-#define _VCS(n) (VCS + (n))
+ enum intel_engine_id id;
unsigned int exec_id;
- enum intel_engine_hw_id {
- RCS_HW = 0,
- VCS_HW,
- BCS_HW,
- VECS_HW,
- VCS2_HW
- } hw_id;
- enum intel_engine_hw_id guc_id; /* XXX same as hw_id? */
+ unsigned int hw_id;
+ unsigned int guc_id;
u32 mmio_base;
unsigned int irq_shift;
struct intel_ring *buffer;
struct intel_render_state *render_state;
+ atomic_t irq_count;
+ unsigned long irq_posted;
+#define ENGINE_IRQ_BREADCRUMB 0
+#define ENGINE_IRQ_EXECLIST 1
+
/* Rather than have every client wait upon all user interrupts,
* with the herd waking after every interrupt and each doing the
* heavyweight seqno dance, we delegate the task (of being the
* the overhead of waking that client is much preferred.
*/
struct intel_breadcrumbs {
- struct task_struct __rcu *irq_seqno_bh; /* bh for interrupts */
- bool irq_posted;
+ spinlock_t irq_lock; /* protects irq_*; irqsafe */
+ struct intel_wait *irq_wait; /* oldest waiter by retirement */
- spinlock_t lock; /* protects the lists of requests; irqsafe */
+ spinlock_t rb_lock; /* protects the rb and wraps irq_lock */
struct rb_root waiters; /* sorted by retirement, priority */
struct rb_root signals; /* sorted by retirement */
- struct intel_wait *first_wait; /* oldest waiter by retirement */
struct task_struct *signaler; /* used for fence signalling */
- struct drm_i915_gem_request *first_signal;
+ struct drm_i915_gem_request __rcu *first_signal;
struct timer_list fake_irq; /* used after a missed interrupt */
struct timer_list hangcheck; /* detect missed interrupts */
- unsigned long timeout;
+ unsigned int hangcheck_interrupts;
+ bool irq_armed : 1;
bool irq_enabled : 1;
- bool rpm_wakelock : 1;
+ I915_SELFTEST_DECLARE(bool mock : 1);
} breadcrumbs;
/*
void (*reset_hw)(struct intel_engine_cs *engine,
struct drm_i915_gem_request *req);
+ void (*set_default_submission)(struct intel_engine_cs *engine);
+
int (*context_pin)(struct intel_engine_cs *engine,
struct i915_gem_context *ctx);
void (*context_unpin)(struct intel_engine_cs *engine,
#define I915_DISPATCH_PINNED BIT(1)
#define I915_DISPATCH_RS BIT(2)
void (*emit_breadcrumb)(struct drm_i915_gem_request *req,
- u32 *out);
+ u32 *cs);
int emit_breadcrumb_sz;
/* Pass the request to the hardware queue (e.g. directly into
/* AKA wait() */
int (*sync_to)(struct drm_i915_gem_request *req,
struct drm_i915_gem_request *signal);
- u32 *(*signal)(struct drm_i915_gem_request *req, u32 *out);
+ u32 *(*signal)(struct drm_i915_gem_request *req, u32 *cs);
} semaphore;
/* Execlists */
struct execlist_port {
struct drm_i915_gem_request *request;
unsigned int count;
+ GEM_DEBUG_DECL(u32 context_id);
} execlist_port[2];
struct rb_root execlist_queue;
struct rb_node *execlist_first;
unsigned int fw_domains;
- bool disable_lite_restore_wa;
- bool preempt_wa;
- u32 ctx_desc_template;
/* Contexts are pinned whilst they are active on the GPU. The last
* context executed remains active whilst the GPU is idle - the
intel_write_status_page(struct intel_engine_cs *engine,
int reg, u32 value)
{
+ mb();
+ clflush(&engine->status_page.page_addr[reg]);
engine->status_page.page_addr[reg] = value;
+ clflush(&engine->status_page.page_addr[reg]);
+ mb();
}
/*
void intel_legacy_submission_resume(struct drm_i915_private *dev_priv);
-int __must_check intel_ring_begin(struct drm_i915_gem_request *req, int n);
int __must_check intel_ring_cacheline_align(struct drm_i915_gem_request *req);
-static inline void intel_ring_emit(struct intel_ring *ring, u32 data)
-{
- *(uint32_t *)(ring->vaddr + ring->tail) = data;
- ring->tail += 4;
-}
+u32 __must_check *intel_ring_begin(struct drm_i915_gem_request *req, int n);
-static inline void intel_ring_emit_reg(struct intel_ring *ring, i915_reg_t reg)
-{
- intel_ring_emit(ring, i915_mmio_reg_offset(reg));
-}
-
-static inline void intel_ring_advance(struct intel_ring *ring)
+static inline void
+intel_ring_advance(struct drm_i915_gem_request *req, u32 *cs)
{
/* Dummy function.
*
* reserved for the command packet (i.e. the value passed to
* intel_ring_begin()).
*/
+ GEM_BUG_ON((req->ring->vaddr + req->ring->tail) != cs);
}
-static inline u32 intel_ring_offset(struct intel_ring *ring, void *addr)
+static inline u32
+intel_ring_offset(struct drm_i915_gem_request *req, void *addr)
{
/* Don't write ring->size (equivalent to 0) as that hangs some GPUs. */
- u32 offset = addr - ring->vaddr;
- return offset & (ring->size - 1);
+ u32 offset = addr - req->ring->vaddr;
+ GEM_BUG_ON(offset > req->ring->size);
+ return offset & (req->ring->size - 1);
}
-int __intel_ring_space(int head, int tail, int size);
void intel_ring_update_space(struct intel_ring *ring);
void intel_engine_init_global_seqno(struct intel_engine_cs *engine, u32 seqno);
* wtih serialising this hint with anything, so document it as
* a hint and nothing more.
*/
- return READ_ONCE(engine->timeline->last_submitted_seqno);
+ return READ_ONCE(engine->timeline->seqno);
}
int init_workarounds_ring(struct intel_engine_cs *engine);
+int intel_ring_workarounds_emit(struct drm_i915_gem_request *req);
void intel_engine_get_instdone(struct intel_engine_cs *engine,
struct intel_instdone *instdone);
/* intel_breadcrumbs.c -- user interrupt bottom-half for waiters */
int intel_engine_init_breadcrumbs(struct intel_engine_cs *engine);
-static inline void intel_wait_init(struct intel_wait *wait, u32 seqno)
+static inline void intel_wait_init(struct intel_wait *wait,
+ struct drm_i915_gem_request *rq)
+{
+ wait->tsk = current;
+ wait->request = rq;
+}
+
+static inline void intel_wait_init_for_seqno(struct intel_wait *wait, u32 seqno)
{
wait->tsk = current;
wait->seqno = seqno;
}
+static inline bool intel_wait_has_seqno(const struct intel_wait *wait)
+{
+ return wait->seqno;
+}
+
+static inline bool
+intel_wait_update_seqno(struct intel_wait *wait, u32 seqno)
+{
+ wait->seqno = seqno;
+ return intel_wait_has_seqno(wait);
+}
+
+static inline bool
+intel_wait_update_request(struct intel_wait *wait,
+ const struct drm_i915_gem_request *rq)
+{
+ return intel_wait_update_seqno(wait, i915_gem_request_global_seqno(rq));
+}
+
+static inline bool
+intel_wait_check_seqno(const struct intel_wait *wait, u32 seqno)
+{
+ return wait->seqno == seqno;
+}
+
+static inline bool
+intel_wait_check_request(const struct intel_wait *wait,
+ const struct drm_i915_gem_request *rq)
+{
+ return intel_wait_check_seqno(wait, i915_gem_request_global_seqno(rq));
+}
+
static inline bool intel_wait_complete(const struct intel_wait *wait)
{
return RB_EMPTY_NODE(&wait->node);
void intel_engine_remove_wait(struct intel_engine_cs *engine,
struct intel_wait *wait);
void intel_engine_enable_signaling(struct drm_i915_gem_request *request);
+void intel_engine_cancel_signaling(struct drm_i915_gem_request *request);
static inline bool intel_engine_has_waiter(const struct intel_engine_cs *engine)
{
- return rcu_access_pointer(engine->breadcrumbs.irq_seqno_bh);
+ return READ_ONCE(engine->breadcrumbs.irq_wait);
}
-static inline bool intel_engine_wakeup(const struct intel_engine_cs *engine)
+unsigned int intel_engine_wakeup(struct intel_engine_cs *engine);
+#define ENGINE_WAKEUP_WAITER BIT(0)
+#define ENGINE_WAKEUP_ASLEEP BIT(1)
+
+void __intel_engine_disarm_breadcrumbs(struct intel_engine_cs *engine);
+void intel_engine_disarm_breadcrumbs(struct intel_engine_cs *engine);
+
+void intel_engine_reset_breadcrumbs(struct intel_engine_cs *engine);
+void intel_engine_fini_breadcrumbs(struct intel_engine_cs *engine);
+bool intel_breadcrumbs_busy(struct intel_engine_cs *engine);
+
+static inline u32 *gen8_emit_pipe_control(u32 *batch, u32 flags, u32 offset)
{
- bool wakeup = false;
+ memset(batch, 0, 6 * sizeof(u32));
- /* Note that for this not to dangerously chase a dangling pointer,
- * we must hold the rcu_read_lock here.
- *
- * Also note that tsk is likely to be in !TASK_RUNNING state so an
- * early test for tsk->state != TASK_RUNNING before wake_up_process()
- * is unlikely to be beneficial.
- */
- if (intel_engine_has_waiter(engine)) {
- struct task_struct *tsk;
-
- rcu_read_lock();
- tsk = rcu_dereference(engine->breadcrumbs.irq_seqno_bh);
- if (tsk)
- wakeup = wake_up_process(tsk);
- rcu_read_unlock();
- }
+ batch[0] = GFX_OP_PIPE_CONTROL(6);
+ batch[1] = flags;
+ batch[2] = offset;
- return wakeup;
+ return batch + 6;
}
-void intel_engine_reset_breadcrumbs(struct intel_engine_cs *engine);
-void intel_engine_fini_breadcrumbs(struct intel_engine_cs *engine);
-unsigned int intel_breadcrumbs_busy(struct drm_i915_private *i915);
+bool intel_engine_is_idle(struct intel_engine_cs *engine);
+bool intel_engines_are_idle(struct drm_i915_private *dev_priv);
+
+void intel_engines_reset_default_submission(struct drm_i915_private *i915);
#endif /* _INTEL_RINGBUFFER_H_ */
* present for a given platform.
*/
-#define for_each_power_well(i, power_well, domain_mask, power_domains) \
- for (i = 0; \
- i < (power_domains)->power_well_count && \
- ((power_well) = &(power_domains)->power_wells[i]); \
- i++) \
- for_each_if ((power_well)->domains & (domain_mask))
-
-#define for_each_power_well_rev(i, power_well, domain_mask, power_domains) \
- for (i = (power_domains)->power_well_count - 1; \
- i >= 0 && ((power_well) = &(power_domains)->power_wells[i]);\
- i--) \
- for_each_if ((power_well)->domains & (domain_mask))
-
bool intel_display_power_well_is_enabled(struct drm_i915_private *dev_priv,
int power_well_id);
return "PORT_DDI_D_LANES";
case POWER_DOMAIN_PORT_DDI_E_LANES:
return "PORT_DDI_E_LANES";
+ case POWER_DOMAIN_PORT_DDI_A_IO:
+ return "PORT_DDI_A_IO";
+ case POWER_DOMAIN_PORT_DDI_B_IO:
+ return "PORT_DDI_B_IO";
+ case POWER_DOMAIN_PORT_DDI_C_IO:
+ return "PORT_DDI_C_IO";
+ case POWER_DOMAIN_PORT_DDI_D_IO:
+ return "PORT_DDI_D_IO";
+ case POWER_DOMAIN_PORT_DDI_E_IO:
+ return "PORT_DDI_E_IO";
case POWER_DOMAIN_PORT_DSI:
return "PORT_DSI";
case POWER_DOMAIN_PORT_CRT:
bool __intel_display_power_is_enabled(struct drm_i915_private *dev_priv,
enum intel_display_power_domain domain)
{
- struct i915_power_domains *power_domains;
struct i915_power_well *power_well;
bool is_enabled;
- int i;
if (dev_priv->pm.suspended)
return false;
- power_domains = &dev_priv->power_domains;
-
is_enabled = true;
- for_each_power_well_rev(i, power_well, BIT(domain), power_domains) {
+ for_each_power_domain_well_rev(dev_priv, power_well, BIT_ULL(domain)) {
if (power_well->always_on)
continue;
}
#define SKL_DISPLAY_POWERWELL_2_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_TRANSCODER_A) | \
- BIT(POWER_DOMAIN_PIPE_B) | \
- BIT(POWER_DOMAIN_TRANSCODER_B) | \
- BIT(POWER_DOMAIN_PIPE_C) | \
- BIT(POWER_DOMAIN_TRANSCODER_C) | \
- BIT(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_D_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_E_LANES) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_AUX_D) | \
- BIT(POWER_DOMAIN_AUDIO) | \
- BIT(POWER_DOMAIN_VGA) | \
- BIT(POWER_DOMAIN_INIT))
-#define SKL_DISPLAY_DDI_A_E_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_A_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_E_LANES) | \
- BIT(POWER_DOMAIN_INIT))
-#define SKL_DISPLAY_DDI_B_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_INIT))
-#define SKL_DISPLAY_DDI_C_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_INIT))
-#define SKL_DISPLAY_DDI_D_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_D_LANES) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_A) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_B) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_C) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_D_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_E_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_AUX_D) | \
+ BIT_ULL(POWER_DOMAIN_AUDIO) | \
+ BIT_ULL(POWER_DOMAIN_VGA) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
+#define SKL_DISPLAY_DDI_IO_A_E_POWER_DOMAINS ( \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_A_IO) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_E_IO) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
+#define SKL_DISPLAY_DDI_IO_B_POWER_DOMAINS ( \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_IO) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
+#define SKL_DISPLAY_DDI_IO_C_POWER_DOMAINS ( \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_IO) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
+#define SKL_DISPLAY_DDI_IO_D_POWER_DOMAINS ( \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_D_IO) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define SKL_DISPLAY_DC_OFF_POWER_DOMAINS ( \
SKL_DISPLAY_POWERWELL_2_POWER_DOMAINS | \
- BIT(POWER_DOMAIN_MODESET) | \
- BIT(POWER_DOMAIN_AUX_A) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_MODESET) | \
+ BIT_ULL(POWER_DOMAIN_AUX_A) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define BXT_DISPLAY_POWERWELL_2_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_TRANSCODER_A) | \
- BIT(POWER_DOMAIN_PIPE_B) | \
- BIT(POWER_DOMAIN_TRANSCODER_B) | \
- BIT(POWER_DOMAIN_PIPE_C) | \
- BIT(POWER_DOMAIN_TRANSCODER_C) | \
- BIT(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_AUDIO) | \
- BIT(POWER_DOMAIN_VGA) | \
- BIT(POWER_DOMAIN_GMBUS) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_A) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_B) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_C) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_AUDIO) | \
+ BIT_ULL(POWER_DOMAIN_VGA) | \
+ BIT_ULL(POWER_DOMAIN_GMBUS) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define BXT_DISPLAY_DC_OFF_POWER_DOMAINS ( \
BXT_DISPLAY_POWERWELL_2_POWER_DOMAINS | \
- BIT(POWER_DOMAIN_MODESET) | \
- BIT(POWER_DOMAIN_AUX_A) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_MODESET) | \
+ BIT_ULL(POWER_DOMAIN_AUX_A) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define BXT_DPIO_CMN_A_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_A_LANES) | \
- BIT(POWER_DOMAIN_AUX_A) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_A_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_A) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define BXT_DPIO_CMN_BC_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define GLK_DISPLAY_POWERWELL_2_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_TRANSCODER_A) | \
- BIT(POWER_DOMAIN_PIPE_B) | \
- BIT(POWER_DOMAIN_TRANSCODER_B) | \
- BIT(POWER_DOMAIN_PIPE_C) | \
- BIT(POWER_DOMAIN_TRANSCODER_C) | \
- BIT(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_AUDIO) | \
- BIT(POWER_DOMAIN_VGA) | \
- BIT(POWER_DOMAIN_INIT))
-#define GLK_DISPLAY_DDI_A_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_A_LANES) | \
- BIT(POWER_DOMAIN_INIT))
-#define GLK_DISPLAY_DDI_B_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_INIT))
-#define GLK_DISPLAY_DDI_C_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_A) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_B) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_C) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_AUDIO) | \
+ BIT_ULL(POWER_DOMAIN_VGA) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
+#define GLK_DISPLAY_DDI_IO_A_POWER_DOMAINS ( \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_A_IO))
+#define GLK_DISPLAY_DDI_IO_B_POWER_DOMAINS ( \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_IO))
+#define GLK_DISPLAY_DDI_IO_C_POWER_DOMAINS ( \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_IO))
#define GLK_DPIO_CMN_A_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_A_LANES) | \
- BIT(POWER_DOMAIN_AUX_A) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_A_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_A) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define GLK_DPIO_CMN_B_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define GLK_DPIO_CMN_C_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define GLK_DISPLAY_AUX_A_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_AUX_A) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_AUX_A) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define GLK_DISPLAY_AUX_B_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define GLK_DISPLAY_AUX_C_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define GLK_DISPLAY_DC_OFF_POWER_DOMAINS ( \
GLK_DISPLAY_POWERWELL_2_POWER_DOMAINS | \
- BIT(POWER_DOMAIN_MODESET) | \
- BIT(POWER_DOMAIN_AUX_A) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_MODESET) | \
+ BIT_ULL(POWER_DOMAIN_AUX_A) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
static void assert_can_enable_dc9(struct drm_i915_private *dev_priv)
{
* other request bits to be set, so WARN for those.
*/
if (power_well_id == SKL_DISP_PW_1 ||
- ((IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) &&
+ (IS_GEN9_BC(dev_priv) &&
power_well_id == SKL_DISP_PW_MISC_IO))
DRM_DEBUG_DRIVER("Clearing auxiliary requests for %s forced on "
"by DMC\n", power_well->name);
static void hsw_power_well_sync_hw(struct drm_i915_private *dev_priv,
struct i915_power_well *power_well)
{
- hsw_set_power_well(dev_priv, power_well, power_well->count > 0);
-
- /*
- * We're taking over the BIOS, so clear any requests made by it since
- * the driver is in charge now.
- */
- if (I915_READ(HSW_PWR_WELL_BIOS) & HSW_PWR_WELL_ENABLE_REQUEST)
+ /* Take over the request bit if set by BIOS. */
+ if (I915_READ(HSW_PWR_WELL_BIOS) & HSW_PWR_WELL_ENABLE_REQUEST) {
+ if (!(I915_READ(HSW_PWR_WELL_DRIVER) &
+ HSW_PWR_WELL_ENABLE_REQUEST))
+ I915_WRITE(HSW_PWR_WELL_DRIVER,
+ HSW_PWR_WELL_ENABLE_REQUEST);
I915_WRITE(HSW_PWR_WELL_BIOS, 0);
+ }
}
static void hsw_power_well_enable(struct drm_i915_private *dev_priv,
static void skl_power_well_sync_hw(struct drm_i915_private *dev_priv,
struct i915_power_well *power_well)
{
- skl_set_power_well(dev_priv, power_well, power_well->count > 0);
+ uint32_t mask = SKL_POWER_WELL_REQ(power_well->id);
+ uint32_t bios_req = I915_READ(HSW_PWR_WELL_BIOS);
+
+ /* Take over the request bit if set by BIOS. */
+ if (bios_req & mask) {
+ uint32_t drv_req = I915_READ(HSW_PWR_WELL_DRIVER);
- /* Clear any request made by BIOS as driver is taking over */
- I915_WRITE(HSW_PWR_WELL_BIOS, 0);
+ if (!(drv_req & mask))
+ I915_WRITE(HSW_PWR_WELL_DRIVER, drv_req | mask);
+ I915_WRITE(HSW_PWR_WELL_BIOS, bios_req & ~mask);
+ }
}
static void skl_power_well_enable(struct drm_i915_private *dev_priv,
return bxt_ddi_phy_is_enabled(dev_priv, power_well->data);
}
-static void bxt_dpio_cmn_power_well_sync_hw(struct drm_i915_private *dev_priv,
- struct i915_power_well *power_well)
-{
- if (power_well->count > 0)
- bxt_dpio_cmn_power_well_enable(dev_priv, power_well);
- else
- bxt_dpio_cmn_power_well_disable(dev_priv, power_well);
-}
-
-
static void bxt_verify_ddi_phy_power_wells(struct drm_i915_private *dev_priv)
{
struct i915_power_well *power_well;
static void gen9_dc_off_power_well_enable(struct drm_i915_private *dev_priv,
struct i915_power_well *power_well)
{
+ struct intel_cdclk_state cdclk_state = {};
+
gen9_set_dc_state(dev_priv, DC_STATE_DISABLE);
- WARN_ON(dev_priv->cdclk_freq !=
- dev_priv->display.get_display_clock_speed(dev_priv));
+ dev_priv->display.get_cdclk(dev_priv, &cdclk_state);
+ WARN_ON(!intel_cdclk_state_compare(&dev_priv->cdclk.hw, &cdclk_state));
gen9_assert_dbuf_enabled(dev_priv);
gen9_enable_dc5(dev_priv);
}
-static void gen9_dc_off_power_well_sync_hw(struct drm_i915_private *dev_priv,
- struct i915_power_well *power_well)
+static void i9xx_power_well_sync_hw_noop(struct drm_i915_private *dev_priv,
+ struct i915_power_well *power_well)
{
- if (power_well->count > 0)
- gen9_dc_off_power_well_enable(dev_priv, power_well);
- else
- gen9_dc_off_power_well_disable(dev_priv, power_well);
}
static void i9xx_always_on_power_well_noop(struct drm_i915_private *dev_priv,
mutex_unlock(&dev_priv->rps.hw_lock);
}
-static void vlv_power_well_sync_hw(struct drm_i915_private *dev_priv,
- struct i915_power_well *power_well)
-{
- vlv_set_power_well(dev_priv, power_well, power_well->count > 0);
-}
-
static void vlv_power_well_enable(struct drm_i915_private *dev_priv,
struct i915_power_well *power_well)
{
vlv_set_power_well(dev_priv, power_well, false);
}
-#define POWER_DOMAIN_MASK (BIT(POWER_DOMAIN_NUM) - 1)
+#define POWER_DOMAIN_MASK (GENMASK_ULL(POWER_DOMAIN_NUM - 1, 0))
static struct i915_power_well *lookup_power_well(struct drm_i915_private *dev_priv,
int power_well_id)
mutex_unlock(&dev_priv->rps.hw_lock);
}
-static void chv_pipe_power_well_sync_hw(struct drm_i915_private *dev_priv,
- struct i915_power_well *power_well)
-{
- WARN_ON_ONCE(power_well->id != PIPE_A);
-
- chv_set_pipe_power_well(dev_priv, power_well, power_well->count > 0);
-}
-
static void chv_pipe_power_well_enable(struct drm_i915_private *dev_priv,
struct i915_power_well *power_well)
{
{
struct i915_power_domains *power_domains = &dev_priv->power_domains;
struct i915_power_well *power_well;
- int i;
- for_each_power_well(i, power_well, BIT(domain), power_domains)
+ for_each_power_domain_well(dev_priv, power_well, BIT_ULL(domain))
intel_power_well_get(dev_priv, power_well);
power_domains->domain_use_count[domain]++;
{
struct i915_power_domains *power_domains;
struct i915_power_well *power_well;
- int i;
power_domains = &dev_priv->power_domains;
intel_display_power_domain_str(domain));
power_domains->domain_use_count[domain]--;
- for_each_power_well_rev(i, power_well, BIT(domain), power_domains)
+ for_each_power_domain_well_rev(dev_priv, power_well, BIT_ULL(domain))
intel_power_well_put(dev_priv, power_well);
mutex_unlock(&power_domains->lock);
}
#define HSW_DISPLAY_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PIPE_B) | \
- BIT(POWER_DOMAIN_PIPE_C) | \
- BIT(POWER_DOMAIN_PIPE_A_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_TRANSCODER_A) | \
- BIT(POWER_DOMAIN_TRANSCODER_B) | \
- BIT(POWER_DOMAIN_TRANSCODER_C) | \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_D_LANES) | \
- BIT(POWER_DOMAIN_PORT_CRT) | /* DDI E */ \
- BIT(POWER_DOMAIN_VGA) | \
- BIT(POWER_DOMAIN_AUDIO) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PIPE_B) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_A_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_A) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_B) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_C) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_D_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_CRT) | /* DDI E */ \
+ BIT_ULL(POWER_DOMAIN_VGA) | \
+ BIT_ULL(POWER_DOMAIN_AUDIO) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define BDW_DISPLAY_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PIPE_B) | \
- BIT(POWER_DOMAIN_PIPE_C) | \
- BIT(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_TRANSCODER_A) | \
- BIT(POWER_DOMAIN_TRANSCODER_B) | \
- BIT(POWER_DOMAIN_TRANSCODER_C) | \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_D_LANES) | \
- BIT(POWER_DOMAIN_PORT_CRT) | /* DDI E */ \
- BIT(POWER_DOMAIN_VGA) | \
- BIT(POWER_DOMAIN_AUDIO) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PIPE_B) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_A) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_B) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_C) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_D_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_CRT) | /* DDI E */ \
+ BIT_ULL(POWER_DOMAIN_VGA) | \
+ BIT_ULL(POWER_DOMAIN_AUDIO) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define VLV_DISPLAY_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PIPE_A) | \
- BIT(POWER_DOMAIN_PIPE_B) | \
- BIT(POWER_DOMAIN_PIPE_A_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_TRANSCODER_A) | \
- BIT(POWER_DOMAIN_TRANSCODER_B) | \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_PORT_DSI) | \
- BIT(POWER_DOMAIN_PORT_CRT) | \
- BIT(POWER_DOMAIN_VGA) | \
- BIT(POWER_DOMAIN_AUDIO) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_GMBUS) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PIPE_A) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_A_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_A) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_B) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DSI) | \
+ BIT_ULL(POWER_DOMAIN_PORT_CRT) | \
+ BIT_ULL(POWER_DOMAIN_VGA) | \
+ BIT_ULL(POWER_DOMAIN_AUDIO) | \
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_GMBUS) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define VLV_DPIO_CMN_BC_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_PORT_CRT) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_CRT) | \
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define VLV_DPIO_TX_B_LANES_01_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define VLV_DPIO_TX_B_LANES_23_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define VLV_DPIO_TX_C_LANES_01_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define VLV_DPIO_TX_C_LANES_23_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define CHV_DISPLAY_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PIPE_A) | \
- BIT(POWER_DOMAIN_PIPE_B) | \
- BIT(POWER_DOMAIN_PIPE_C) | \
- BIT(POWER_DOMAIN_PIPE_A_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_TRANSCODER_A) | \
- BIT(POWER_DOMAIN_TRANSCODER_B) | \
- BIT(POWER_DOMAIN_TRANSCODER_C) | \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_D_LANES) | \
- BIT(POWER_DOMAIN_PORT_DSI) | \
- BIT(POWER_DOMAIN_VGA) | \
- BIT(POWER_DOMAIN_AUDIO) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_AUX_D) | \
- BIT(POWER_DOMAIN_GMBUS) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PIPE_A) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_A_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_A) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_B) | \
+ BIT_ULL(POWER_DOMAIN_TRANSCODER_C) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_D_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DSI) | \
+ BIT_ULL(POWER_DOMAIN_VGA) | \
+ BIT_ULL(POWER_DOMAIN_AUDIO) | \
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_AUX_D) | \
+ BIT_ULL(POWER_DOMAIN_GMBUS) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define CHV_DPIO_CMN_BC_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
- BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_B) | \
+ BIT_ULL(POWER_DOMAIN_AUX_C) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
#define CHV_DPIO_CMN_D_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PORT_DDI_D_LANES) | \
- BIT(POWER_DOMAIN_AUX_D) | \
- BIT(POWER_DOMAIN_INIT))
+ BIT_ULL(POWER_DOMAIN_PORT_DDI_D_LANES) | \
+ BIT_ULL(POWER_DOMAIN_AUX_D) | \
+ BIT_ULL(POWER_DOMAIN_INIT))
static const struct i915_power_well_ops i9xx_always_on_power_well_ops = {
- .sync_hw = i9xx_always_on_power_well_noop,
+ .sync_hw = i9xx_power_well_sync_hw_noop,
.enable = i9xx_always_on_power_well_noop,
.disable = i9xx_always_on_power_well_noop,
.is_enabled = i9xx_always_on_power_well_enabled,
};
static const struct i915_power_well_ops chv_pipe_power_well_ops = {
- .sync_hw = chv_pipe_power_well_sync_hw,
+ .sync_hw = i9xx_power_well_sync_hw_noop,
.enable = chv_pipe_power_well_enable,
.disable = chv_pipe_power_well_disable,
.is_enabled = chv_pipe_power_well_enabled,
};
static const struct i915_power_well_ops chv_dpio_cmn_power_well_ops = {
- .sync_hw = vlv_power_well_sync_hw,
+ .sync_hw = i9xx_power_well_sync_hw_noop,
.enable = chv_dpio_cmn_power_well_enable,
.disable = chv_dpio_cmn_power_well_disable,
.is_enabled = vlv_power_well_enabled,
};
static const struct i915_power_well_ops gen9_dc_off_power_well_ops = {
- .sync_hw = gen9_dc_off_power_well_sync_hw,
+ .sync_hw = i9xx_power_well_sync_hw_noop,
.enable = gen9_dc_off_power_well_enable,
.disable = gen9_dc_off_power_well_disable,
.is_enabled = gen9_dc_off_power_well_enabled,
};
static const struct i915_power_well_ops bxt_dpio_cmn_power_well_ops = {
- .sync_hw = bxt_dpio_cmn_power_well_sync_hw,
+ .sync_hw = i9xx_power_well_sync_hw_noop,
.enable = bxt_dpio_cmn_power_well_enable,
.disable = bxt_dpio_cmn_power_well_disable,
.is_enabled = bxt_dpio_cmn_power_well_enabled,
};
static const struct i915_power_well_ops vlv_display_power_well_ops = {
- .sync_hw = vlv_power_well_sync_hw,
+ .sync_hw = i9xx_power_well_sync_hw_noop,
.enable = vlv_display_power_well_enable,
.disable = vlv_display_power_well_disable,
.is_enabled = vlv_power_well_enabled,
};
static const struct i915_power_well_ops vlv_dpio_cmn_power_well_ops = {
- .sync_hw = vlv_power_well_sync_hw,
+ .sync_hw = i9xx_power_well_sync_hw_noop,
.enable = vlv_dpio_cmn_power_well_enable,
.disable = vlv_dpio_cmn_power_well_disable,
.is_enabled = vlv_power_well_enabled,
};
static const struct i915_power_well_ops vlv_dpio_power_well_ops = {
- .sync_hw = vlv_power_well_sync_hw,
+ .sync_hw = i9xx_power_well_sync_hw_noop,
.enable = vlv_power_well_enable,
.disable = vlv_power_well_disable,
.is_enabled = vlv_power_well_enabled,
.id = SKL_DISP_PW_2,
},
{
- .name = "DDI A/E power well",
- .domains = SKL_DISPLAY_DDI_A_E_POWER_DOMAINS,
+ .name = "DDI A/E IO power well",
+ .domains = SKL_DISPLAY_DDI_IO_A_E_POWER_DOMAINS,
.ops = &skl_power_well_ops,
.id = SKL_DISP_PW_DDI_A_E,
},
{
- .name = "DDI B power well",
- .domains = SKL_DISPLAY_DDI_B_POWER_DOMAINS,
+ .name = "DDI B IO power well",
+ .domains = SKL_DISPLAY_DDI_IO_B_POWER_DOMAINS,
.ops = &skl_power_well_ops,
.id = SKL_DISP_PW_DDI_B,
},
{
- .name = "DDI C power well",
- .domains = SKL_DISPLAY_DDI_C_POWER_DOMAINS,
+ .name = "DDI C IO power well",
+ .domains = SKL_DISPLAY_DDI_IO_C_POWER_DOMAINS,
.ops = &skl_power_well_ops,
.id = SKL_DISP_PW_DDI_C,
},
{
- .name = "DDI D power well",
- .domains = SKL_DISPLAY_DDI_D_POWER_DOMAINS,
+ .name = "DDI D IO power well",
+ .domains = SKL_DISPLAY_DDI_IO_D_POWER_DOMAINS,
.ops = &skl_power_well_ops,
.id = SKL_DISP_PW_DDI_D,
},
.id = GLK_DISP_PW_AUX_C,
},
{
- .name = "DDI A power well",
- .domains = GLK_DISPLAY_DDI_A_POWER_DOMAINS,
+ .name = "DDI A IO power well",
+ .domains = GLK_DISPLAY_DDI_IO_A_POWER_DOMAINS,
.ops = &skl_power_well_ops,
.id = GLK_DISP_PW_DDI_A,
},
{
- .name = "DDI B power well",
- .domains = GLK_DISPLAY_DDI_B_POWER_DOMAINS,
+ .name = "DDI B IO power well",
+ .domains = GLK_DISPLAY_DDI_IO_B_POWER_DOMAINS,
.ops = &skl_power_well_ops,
.id = SKL_DISP_PW_DDI_B,
},
{
- .name = "DDI C power well",
- .domains = GLK_DISPLAY_DDI_C_POWER_DOMAINS,
+ .name = "DDI C IO power well",
+ .domains = GLK_DISPLAY_DDI_IO_C_POWER_DOMAINS,
.ops = &skl_power_well_ops,
.id = SKL_DISP_PW_DDI_C,
},
int requested_dc;
int max_dc;
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ if (IS_GEN9_BC(dev_priv)) {
max_dc = 2;
mask = 0;
} else if (IS_GEN9_LP(dev_priv)) {
dev_priv->csr.allowed_dc_mask = get_allowed_dc_mask(dev_priv,
i915.enable_dc);
- BUILD_BUG_ON(POWER_DOMAIN_NUM > 31);
+ BUILD_BUG_ON(POWER_DOMAIN_NUM > 64);
mutex_init(&power_domains->lock);
set_power_wells(power_domains, hsw_power_wells);
} else if (IS_BROADWELL(dev_priv)) {
set_power_wells(power_domains, bdw_power_wells);
- } else if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ } else if (IS_GEN9_BC(dev_priv)) {
set_power_wells(power_domains, skl_power_wells);
} else if (IS_BROXTON(dev_priv)) {
set_power_wells(power_domains, bxt_power_wells);
{
struct i915_power_domains *power_domains = &dev_priv->power_domains;
struct i915_power_well *power_well;
- int i;
mutex_lock(&power_domains->lock);
- for_each_power_well(i, power_well, POWER_DOMAIN_MASK, power_domains) {
+ for_each_power_well(dev_priv, power_well) {
power_well->ops->sync_hw(dev_priv, power_well);
power_well->hw_enabled = power_well->ops->is_enabled(dev_priv,
power_well);
* @resume: Called from resume code paths or not
*
* This function initializes the hardware power domain state and enables all
- * power domains using intel_display_set_init_power().
+ * power wells belonging to the INIT power domain. Power wells in other
+ * domains (and not in the INIT domain) are referenced or disabled during the
+ * modeset state HW readout. After that the reference count of each power well
+ * must match its HW enabled state, see intel_power_domains_verify_state().
*/
void intel_power_domains_init_hw(struct drm_i915_private *dev_priv, bool resume)
{
power_domains->initializing = true;
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ if (IS_GEN9_BC(dev_priv)) {
skl_display_core_init(dev_priv, resume);
} else if (IS_GEN9_LP(dev_priv)) {
bxt_display_core_init(dev_priv, resume);
if (!i915.disable_power_well)
intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
- if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
+ if (IS_GEN9_BC(dev_priv))
skl_display_core_uninit(dev_priv);
else if (IS_GEN9_LP(dev_priv))
bxt_display_core_uninit(dev_priv);
}
+static void intel_power_domains_dump_info(struct drm_i915_private *dev_priv)
+{
+ struct i915_power_domains *power_domains = &dev_priv->power_domains;
+ struct i915_power_well *power_well;
+
+ for_each_power_well(dev_priv, power_well) {
+ enum intel_display_power_domain domain;
+
+ DRM_DEBUG_DRIVER("%-25s %d\n",
+ power_well->name, power_well->count);
+
+ for_each_power_domain(domain, power_well->domains)
+ DRM_DEBUG_DRIVER(" %-23s %d\n",
+ intel_display_power_domain_str(domain),
+ power_domains->domain_use_count[domain]);
+ }
+}
+
+/**
+ * intel_power_domains_verify_state - verify the HW/SW state for all power wells
+ * @dev_priv: i915 device instance
+ *
+ * Verify if the reference count of each power well matches its HW enabled
+ * state and the total refcount of the domains it belongs to. This must be
+ * called after modeset HW state sanitization, which is responsible for
+ * acquiring reference counts for any power wells in use and disabling the
+ * ones left on by BIOS but not required by any active output.
+ */
+void intel_power_domains_verify_state(struct drm_i915_private *dev_priv)
+{
+ struct i915_power_domains *power_domains = &dev_priv->power_domains;
+ struct i915_power_well *power_well;
+ bool dump_domain_info;
+
+ mutex_lock(&power_domains->lock);
+
+ dump_domain_info = false;
+ for_each_power_well(dev_priv, power_well) {
+ enum intel_display_power_domain domain;
+ int domains_count;
+ bool enabled;
+
+ /*
+ * Power wells not belonging to any domain (like the MISC_IO
+ * and PW1 power wells) are under FW control, so ignore them,
+ * since their state can change asynchronously.
+ */
+ if (!power_well->domains)
+ continue;
+
+ enabled = power_well->ops->is_enabled(dev_priv, power_well);
+ if ((power_well->count || power_well->always_on) != enabled)
+ DRM_ERROR("power well %s state mismatch (refcount %d/enabled %d)",
+ power_well->name, power_well->count, enabled);
+
+ domains_count = 0;
+ for_each_power_domain(domain, power_well->domains)
+ domains_count += power_domains->domain_use_count[domain];
+
+ if (power_well->count != domains_count) {
+ DRM_ERROR("power well %s refcount/domain refcount mismatch "
+ "(refcount %d/domains refcount %d)\n",
+ power_well->name, power_well->count,
+ domains_count);
+ dump_domain_info = true;
+ }
+ }
+
+ if (dump_domain_info) {
+ static bool dumped;
+
+ if (!dumped) {
+ intel_power_domains_dump_info(dev_priv);
+ dumped = true;
+ }
+ }
+
+ mutex_unlock(&power_domains->lock);
+}
+
/**
* intel_runtime_pm_get - grab a runtime pm reference
* @dev_priv: i915 device instance
/* encoder type will be decided later */
intel_encoder = &intel_sdvo->base;
intel_encoder->type = INTEL_OUTPUT_SDVO;
+ intel_encoder->power_domain = POWER_DOMAIN_PORT_OTHER;
intel_encoder->port = port;
drm_encoder_init(&dev_priv->drm, &intel_encoder->base,
&intel_sdvo_enc_funcs, 0,
}
I915_WRITE(VLV_IOSF_ADDR, addr);
- if (!is_read)
- I915_WRITE(VLV_IOSF_DATA, *val);
+ I915_WRITE(VLV_IOSF_DATA, is_read ? 0 : *val);
I915_WRITE(VLV_IOSF_DOORBELL_REQ, cmd);
if (intel_wait_for_register(dev_priv,
if (is_read)
*val = I915_READ(VLV_IOSF_DATA);
- I915_WRITE(VLV_IOSF_DATA, 0);
return 0;
}
return val;
}
-void vlv_punit_write(struct drm_i915_private *dev_priv, u32 addr, u32 val)
+int vlv_punit_write(struct drm_i915_private *dev_priv, u32 addr, u32 val)
{
+ int err;
+
WARN_ON(!mutex_is_locked(&dev_priv->rps.hw_lock));
mutex_lock(&dev_priv->sb_lock);
- vlv_sideband_rw(dev_priv, PCI_DEVFN(0, 0), IOSF_PORT_PUNIT,
- SB_CRWRDA_NP, addr, &val);
+ err = vlv_sideband_rw(dev_priv, PCI_DEVFN(0, 0), IOSF_PORT_PUNIT,
+ SB_CRWRDA_NP, addr, &val);
mutex_unlock(&dev_priv->sb_lock);
+
+ return err;
}
u32 vlv_bunit_read(struct drm_i915_private *dev_priv, u32 reg)
}
I915_WRITE(SBI_ADDR, (reg << 16));
+ I915_WRITE(SBI_DATA, 0);
if (destination == SBI_ICLK)
value = SBI_CTL_DEST_ICLK | SBI_CTL_OP_CRRD;
if (intel_wait_for_register(dev_priv,
SBI_CTL_STAT,
- SBI_BUSY | SBI_RESPONSE_FAIL,
+ SBI_BUSY,
0,
100)) {
- DRM_ERROR("timeout waiting for SBI to complete read transaction\n");
+ DRM_ERROR("timeout waiting for SBI to complete read\n");
+ return 0;
+ }
+
+ if (I915_READ(SBI_CTL_STAT) & SBI_RESPONSE_FAIL) {
+ DRM_ERROR("error during SBI read of reg %x\n", reg);
return 0;
}
if (intel_wait_for_register(dev_priv,
SBI_CTL_STAT,
- SBI_BUSY | SBI_RESPONSE_FAIL,
+ SBI_BUSY,
0,
100)) {
- DRM_ERROR("timeout waiting for SBI to complete write transaction\n");
+ DRM_ERROR("timeout waiting for SBI to complete write\n");
+ return;
+ }
+
+ if (I915_READ(SBI_CTL_STAT) & SBI_RESPONSE_FAIL) {
+ DRM_ERROR("error during SBI write of %x to reg %x\n",
+ value, reg);
return;
}
}
1000 * adjusted_mode->crtc_htotal);
}
+#define VBLANK_EVASION_TIME_US 100
+
/**
* intel_pipe_update_start() - start update of a set of display registers
* @crtc: the crtc of which the registers are going to be updated
vblank_start = DIV_ROUND_UP(vblank_start, 2);
/* FIXME needs to be calibrated sensibly */
- min = vblank_start - intel_usecs_to_scanlines(adjusted_mode, 100);
+ min = vblank_start - intel_usecs_to_scanlines(adjusted_mode,
+ VBLANK_EVASION_TIME_US);
max = vblank_start - 1;
local_irq_disable();
int scanline_end = intel_get_crtc_scanline(crtc);
u32 end_vbl_count = intel_crtc_get_vblank_counter(crtc);
ktime_t end_vbl_time = ktime_get();
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
if (work) {
work->flip_queued_vblank = end_vbl_count;
local_irq_enable();
+ if (intel_vgpu_active(dev_priv))
+ return;
+
if (crtc->debug.start_vbl_count &&
crtc->debug.start_vbl_count != end_vbl_count) {
DRM_ERROR("Atomic update failure on pipe %c (start=%u end=%u) time %lld us, min %d, max %d, scanline start %d, end %d\n",
ktime_us_delta(end_vbl_time, crtc->debug.start_vbl_time),
crtc->debug.min_vbl, crtc->debug.max_vbl,
crtc->debug.scanline_start, scanline_end);
- }
+ } else if (ktime_us_delta(end_vbl_time, crtc->debug.start_vbl_time) >
+ VBLANK_EVASION_TIME_US)
+ DRM_WARN("Atomic update on pipe (%c) took %lld us, max time under evasion is %u us\n",
+ pipe_name(pipe),
+ ktime_us_delta(end_vbl_time, crtc->debug.start_vbl_time),
+ VBLANK_EVASION_TIME_US);
}
static void
uint32_t y = plane_state->main.y;
uint32_t src_w = drm_rect_width(&plane_state->base.src) >> 16;
uint32_t src_h = drm_rect_height(&plane_state->base.src) >> 16;
+ unsigned long irqflags;
- plane_ctl = PLANE_CTL_ENABLE |
- PLANE_CTL_PIPE_GAMMA_ENABLE |
- PLANE_CTL_PIPE_CSC_ENABLE;
+ plane_ctl = PLANE_CTL_ENABLE;
+
+ if (!IS_GEMINILAKE(dev_priv)) {
+ plane_ctl |=
+ PLANE_CTL_PIPE_GAMMA_ENABLE |
+ PLANE_CTL_PIPE_CSC_ENABLE |
+ PLANE_CTL_PLANE_GAMMA_DISABLE;
+ }
plane_ctl |= skl_plane_ctl_format(fb->format->format);
plane_ctl |= skl_plane_ctl_tiling(fb->modifier);
-
plane_ctl |= skl_plane_ctl_rotation(rotation);
- if (key->flags) {
- I915_WRITE(PLANE_KEYVAL(pipe, plane_id), key->min_value);
- I915_WRITE(PLANE_KEYMAX(pipe, plane_id), key->max_value);
- I915_WRITE(PLANE_KEYMSK(pipe, plane_id), key->channel_mask);
- }
-
if (key->flags & I915_SET_COLORKEY_DESTINATION)
plane_ctl |= PLANE_CTL_KEY_ENABLE_DESTINATION;
else if (key->flags & I915_SET_COLORKEY_SOURCE)
crtc_w--;
crtc_h--;
- I915_WRITE(PLANE_OFFSET(pipe, plane_id), (y << 16) | x);
- I915_WRITE(PLANE_STRIDE(pipe, plane_id), stride);
- I915_WRITE(PLANE_SIZE(pipe, plane_id), (src_h << 16) | src_w);
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
+ if (IS_GEMINILAKE(dev_priv)) {
+ I915_WRITE_FW(PLANE_COLOR_CTL(pipe, plane_id),
+ PLANE_COLOR_PIPE_GAMMA_ENABLE |
+ PLANE_COLOR_PIPE_CSC_ENABLE |
+ PLANE_COLOR_PLANE_GAMMA_DISABLE);
+ }
+
+ if (key->flags) {
+ I915_WRITE_FW(PLANE_KEYVAL(pipe, plane_id), key->min_value);
+ I915_WRITE_FW(PLANE_KEYMAX(pipe, plane_id), key->max_value);
+ I915_WRITE_FW(PLANE_KEYMSK(pipe, plane_id), key->channel_mask);
+ }
+
+ I915_WRITE_FW(PLANE_OFFSET(pipe, plane_id), (y << 16) | x);
+ I915_WRITE_FW(PLANE_STRIDE(pipe, plane_id), stride);
+ I915_WRITE_FW(PLANE_SIZE(pipe, plane_id), (src_h << 16) | src_w);
/* program plane scaler */
if (plane_state->scaler_id >= 0) {
scaler = &crtc_state->scaler_state.scalers[scaler_id];
- I915_WRITE(SKL_PS_CTRL(pipe, scaler_id),
- PS_SCALER_EN | PS_PLANE_SEL(plane_id) | scaler->mode);
- I915_WRITE(SKL_PS_PWR_GATE(pipe, scaler_id), 0);
- I915_WRITE(SKL_PS_WIN_POS(pipe, scaler_id), (crtc_x << 16) | crtc_y);
- I915_WRITE(SKL_PS_WIN_SZ(pipe, scaler_id),
- ((crtc_w + 1) << 16)|(crtc_h + 1));
+ I915_WRITE_FW(SKL_PS_CTRL(pipe, scaler_id),
+ PS_SCALER_EN | PS_PLANE_SEL(plane_id) | scaler->mode);
+ I915_WRITE_FW(SKL_PS_PWR_GATE(pipe, scaler_id), 0);
+ I915_WRITE_FW(SKL_PS_WIN_POS(pipe, scaler_id), (crtc_x << 16) | crtc_y);
+ I915_WRITE_FW(SKL_PS_WIN_SZ(pipe, scaler_id),
+ ((crtc_w + 1) << 16)|(crtc_h + 1));
- I915_WRITE(PLANE_POS(pipe, plane_id), 0);
+ I915_WRITE_FW(PLANE_POS(pipe, plane_id), 0);
} else {
- I915_WRITE(PLANE_POS(pipe, plane_id), (crtc_y << 16) | crtc_x);
+ I915_WRITE_FW(PLANE_POS(pipe, plane_id), (crtc_y << 16) | crtc_x);
}
- I915_WRITE(PLANE_CTL(pipe, plane_id), plane_ctl);
- I915_WRITE(PLANE_SURF(pipe, plane_id),
- intel_plane_ggtt_offset(plane_state) + surf_addr);
- POSTING_READ(PLANE_SURF(pipe, plane_id));
+ I915_WRITE_FW(PLANE_CTL(pipe, plane_id), plane_ctl);
+ I915_WRITE_FW(PLANE_SURF(pipe, plane_id),
+ intel_plane_ggtt_offset(plane_state) + surf_addr);
+ POSTING_READ_FW(PLANE_SURF(pipe, plane_id));
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void
struct intel_plane *intel_plane = to_intel_plane(dplane);
enum plane_id plane_id = intel_plane->id;
enum pipe pipe = intel_plane->pipe;
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
- I915_WRITE(PLANE_CTL(pipe, plane_id), 0);
+ I915_WRITE_FW(PLANE_CTL(pipe, plane_id), 0);
- I915_WRITE(PLANE_SURF(pipe, plane_id), 0);
- POSTING_READ(PLANE_SURF(pipe, plane_id));
+ I915_WRITE_FW(PLANE_SURF(pipe, plane_id), 0);
+ POSTING_READ_FW(PLANE_SURF(pipe, plane_id));
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void
* Cb and Cr apparently come in as signed already, so no
* need for any offset. For Y we need to remove the offset.
*/
- I915_WRITE(SPCSCYGOFF(plane_id), SPCSC_OOFF(0) | SPCSC_IOFF(-64));
- I915_WRITE(SPCSCCBOFF(plane_id), SPCSC_OOFF(0) | SPCSC_IOFF(0));
- I915_WRITE(SPCSCCROFF(plane_id), SPCSC_OOFF(0) | SPCSC_IOFF(0));
-
- I915_WRITE(SPCSCC01(plane_id), SPCSC_C1(4769) | SPCSC_C0(6537));
- I915_WRITE(SPCSCC23(plane_id), SPCSC_C1(-3330) | SPCSC_C0(0));
- I915_WRITE(SPCSCC45(plane_id), SPCSC_C1(-1605) | SPCSC_C0(4769));
- I915_WRITE(SPCSCC67(plane_id), SPCSC_C1(4769) | SPCSC_C0(0));
- I915_WRITE(SPCSCC8(plane_id), SPCSC_C0(8263));
-
- I915_WRITE(SPCSCYGICLAMP(plane_id), SPCSC_IMAX(940) | SPCSC_IMIN(64));
- I915_WRITE(SPCSCCBICLAMP(plane_id), SPCSC_IMAX(448) | SPCSC_IMIN(-448));
- I915_WRITE(SPCSCCRICLAMP(plane_id), SPCSC_IMAX(448) | SPCSC_IMIN(-448));
-
- I915_WRITE(SPCSCYGOCLAMP(plane_id), SPCSC_OMAX(1023) | SPCSC_OMIN(0));
- I915_WRITE(SPCSCCBOCLAMP(plane_id), SPCSC_OMAX(1023) | SPCSC_OMIN(0));
- I915_WRITE(SPCSCCROCLAMP(plane_id), SPCSC_OMAX(1023) | SPCSC_OMIN(0));
+ I915_WRITE_FW(SPCSCYGOFF(plane_id), SPCSC_OOFF(0) | SPCSC_IOFF(-64));
+ I915_WRITE_FW(SPCSCCBOFF(plane_id), SPCSC_OOFF(0) | SPCSC_IOFF(0));
+ I915_WRITE_FW(SPCSCCROFF(plane_id), SPCSC_OOFF(0) | SPCSC_IOFF(0));
+
+ I915_WRITE_FW(SPCSCC01(plane_id), SPCSC_C1(4769) | SPCSC_C0(6537));
+ I915_WRITE_FW(SPCSCC23(plane_id), SPCSC_C1(-3330) | SPCSC_C0(0));
+ I915_WRITE_FW(SPCSCC45(plane_id), SPCSC_C1(-1605) | SPCSC_C0(4769));
+ I915_WRITE_FW(SPCSCC67(plane_id), SPCSC_C1(4769) | SPCSC_C0(0));
+ I915_WRITE_FW(SPCSCC8(plane_id), SPCSC_C0(8263));
+
+ I915_WRITE_FW(SPCSCYGICLAMP(plane_id), SPCSC_IMAX(940) | SPCSC_IMIN(64));
+ I915_WRITE_FW(SPCSCCBICLAMP(plane_id), SPCSC_IMAX(448) | SPCSC_IMIN(-448));
+ I915_WRITE_FW(SPCSCCRICLAMP(plane_id), SPCSC_IMAX(448) | SPCSC_IMIN(-448));
+
+ I915_WRITE_FW(SPCSCYGOCLAMP(plane_id), SPCSC_OMAX(1023) | SPCSC_OMIN(0));
+ I915_WRITE_FW(SPCSCCBOCLAMP(plane_id), SPCSC_OMAX(1023) | SPCSC_OMIN(0));
+ I915_WRITE_FW(SPCSCCROCLAMP(plane_id), SPCSC_OMAX(1023) | SPCSC_OMIN(0));
}
static void
uint32_t y = plane_state->base.src.y1 >> 16;
uint32_t src_w = drm_rect_width(&plane_state->base.src) >> 16;
uint32_t src_h = drm_rect_height(&plane_state->base.src) >> 16;
+ unsigned long irqflags;
sprctl = SP_ENABLE;
if (rotation & DRM_REFLECT_X)
sprctl |= SP_MIRROR;
+ if (key->flags & I915_SET_COLORKEY_SOURCE)
+ sprctl |= SP_SOURCE_KEY;
+
/* Sizes are 0 based */
src_w--;
src_h--;
linear_offset = intel_fb_xy_to_linear(x, y, plane_state, 0);
- if (key->flags) {
- I915_WRITE(SPKEYMINVAL(pipe, plane_id), key->min_value);
- I915_WRITE(SPKEYMAXVAL(pipe, plane_id), key->max_value);
- I915_WRITE(SPKEYMSK(pipe, plane_id), key->channel_mask);
- }
-
- if (key->flags & I915_SET_COLORKEY_SOURCE)
- sprctl |= SP_SOURCE_KEY;
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
if (IS_CHERRYVIEW(dev_priv) && pipe == PIPE_B)
chv_update_csc(intel_plane, fb->format->format);
- I915_WRITE(SPSTRIDE(pipe, plane_id), fb->pitches[0]);
- I915_WRITE(SPPOS(pipe, plane_id), (crtc_y << 16) | crtc_x);
+ if (key->flags) {
+ I915_WRITE_FW(SPKEYMINVAL(pipe, plane_id), key->min_value);
+ I915_WRITE_FW(SPKEYMAXVAL(pipe, plane_id), key->max_value);
+ I915_WRITE_FW(SPKEYMSK(pipe, plane_id), key->channel_mask);
+ }
+ I915_WRITE_FW(SPSTRIDE(pipe, plane_id), fb->pitches[0]);
+ I915_WRITE_FW(SPPOS(pipe, plane_id), (crtc_y << 16) | crtc_x);
if (fb->modifier == I915_FORMAT_MOD_X_TILED)
- I915_WRITE(SPTILEOFF(pipe, plane_id), (y << 16) | x);
+ I915_WRITE_FW(SPTILEOFF(pipe, plane_id), (y << 16) | x);
else
- I915_WRITE(SPLINOFF(pipe, plane_id), linear_offset);
+ I915_WRITE_FW(SPLINOFF(pipe, plane_id), linear_offset);
+
+ I915_WRITE_FW(SPCONSTALPHA(pipe, plane_id), 0);
- I915_WRITE(SPCONSTALPHA(pipe, plane_id), 0);
+ I915_WRITE_FW(SPSIZE(pipe, plane_id), (crtc_h << 16) | crtc_w);
+ I915_WRITE_FW(SPCNTR(pipe, plane_id), sprctl);
+ I915_WRITE_FW(SPSURF(pipe, plane_id),
+ intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
+ POSTING_READ_FW(SPSURF(pipe, plane_id));
- I915_WRITE(SPSIZE(pipe, plane_id), (crtc_h << 16) | crtc_w);
- I915_WRITE(SPCNTR(pipe, plane_id), sprctl);
- I915_WRITE(SPSURF(pipe, plane_id),
- intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
- POSTING_READ(SPSURF(pipe, plane_id));
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void
struct intel_plane *intel_plane = to_intel_plane(dplane);
enum pipe pipe = intel_plane->pipe;
enum plane_id plane_id = intel_plane->id;
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
- I915_WRITE(SPCNTR(pipe, plane_id), 0);
+ I915_WRITE_FW(SPCNTR(pipe, plane_id), 0);
- I915_WRITE(SPSURF(pipe, plane_id), 0);
- POSTING_READ(SPSURF(pipe, plane_id));
+ I915_WRITE_FW(SPSURF(pipe, plane_id), 0);
+ POSTING_READ_FW(SPSURF(pipe, plane_id));
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void
uint32_t y = plane_state->base.src.y1 >> 16;
uint32_t src_w = drm_rect_width(&plane_state->base.src) >> 16;
uint32_t src_h = drm_rect_height(&plane_state->base.src) >> 16;
+ unsigned long irqflags;
sprctl = SPRITE_ENABLE;
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
sprctl |= SPRITE_PIPE_CSC_ENABLE;
+ if (key->flags & I915_SET_COLORKEY_DESTINATION)
+ sprctl |= SPRITE_DEST_KEY;
+ else if (key->flags & I915_SET_COLORKEY_SOURCE)
+ sprctl |= SPRITE_SOURCE_KEY;
+
/* Sizes are 0 based */
src_w--;
src_h--;
linear_offset = intel_fb_xy_to_linear(x, y, plane_state, 0);
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
if (key->flags) {
- I915_WRITE(SPRKEYVAL(pipe), key->min_value);
- I915_WRITE(SPRKEYMAX(pipe), key->max_value);
- I915_WRITE(SPRKEYMSK(pipe), key->channel_mask);
+ I915_WRITE_FW(SPRKEYVAL(pipe), key->min_value);
+ I915_WRITE_FW(SPRKEYMAX(pipe), key->max_value);
+ I915_WRITE_FW(SPRKEYMSK(pipe), key->channel_mask);
}
- if (key->flags & I915_SET_COLORKEY_DESTINATION)
- sprctl |= SPRITE_DEST_KEY;
- else if (key->flags & I915_SET_COLORKEY_SOURCE)
- sprctl |= SPRITE_SOURCE_KEY;
-
- I915_WRITE(SPRSTRIDE(pipe), fb->pitches[0]);
- I915_WRITE(SPRPOS(pipe), (crtc_y << 16) | crtc_x);
+ I915_WRITE_FW(SPRSTRIDE(pipe), fb->pitches[0]);
+ I915_WRITE_FW(SPRPOS(pipe), (crtc_y << 16) | crtc_x);
/* HSW consolidates SPRTILEOFF and SPRLINOFF into a single SPROFFSET
* register */
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
- I915_WRITE(SPROFFSET(pipe), (y << 16) | x);
+ I915_WRITE_FW(SPROFFSET(pipe), (y << 16) | x);
else if (fb->modifier == I915_FORMAT_MOD_X_TILED)
- I915_WRITE(SPRTILEOFF(pipe), (y << 16) | x);
+ I915_WRITE_FW(SPRTILEOFF(pipe), (y << 16) | x);
else
- I915_WRITE(SPRLINOFF(pipe), linear_offset);
+ I915_WRITE_FW(SPRLINOFF(pipe), linear_offset);
- I915_WRITE(SPRSIZE(pipe), (crtc_h << 16) | crtc_w);
+ I915_WRITE_FW(SPRSIZE(pipe), (crtc_h << 16) | crtc_w);
if (intel_plane->can_scale)
- I915_WRITE(SPRSCALE(pipe), sprscale);
- I915_WRITE(SPRCTL(pipe), sprctl);
- I915_WRITE(SPRSURF(pipe),
- intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
- POSTING_READ(SPRSURF(pipe));
+ I915_WRITE_FW(SPRSCALE(pipe), sprscale);
+ I915_WRITE_FW(SPRCTL(pipe), sprctl);
+ I915_WRITE_FW(SPRSURF(pipe),
+ intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
+ POSTING_READ_FW(SPRSURF(pipe));
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_plane *intel_plane = to_intel_plane(plane);
int pipe = intel_plane->pipe;
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
- I915_WRITE(SPRCTL(pipe), 0);
+ I915_WRITE_FW(SPRCTL(pipe), 0);
/* Can't leave the scaler enabled... */
if (intel_plane->can_scale)
- I915_WRITE(SPRSCALE(pipe), 0);
+ I915_WRITE_FW(SPRSCALE(pipe), 0);
+
+ I915_WRITE_FW(SPRSURF(pipe), 0);
+ POSTING_READ_FW(SPRSURF(pipe));
- I915_WRITE(SPRSURF(pipe), 0);
- POSTING_READ(SPRSURF(pipe));
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void
uint32_t y = plane_state->base.src.y1 >> 16;
uint32_t src_w = drm_rect_width(&plane_state->base.src) >> 16;
uint32_t src_h = drm_rect_height(&plane_state->base.src) >> 16;
+ unsigned long irqflags;
dvscntr = DVS_ENABLE;
if (IS_GEN6(dev_priv))
dvscntr |= DVS_TRICKLE_FEED_DISABLE; /* must disable */
+ if (key->flags & I915_SET_COLORKEY_DESTINATION)
+ dvscntr |= DVS_DEST_KEY;
+ else if (key->flags & I915_SET_COLORKEY_SOURCE)
+ dvscntr |= DVS_SOURCE_KEY;
+
/* Sizes are 0 based */
src_w--;
src_h--;
linear_offset = intel_fb_xy_to_linear(x, y, plane_state, 0);
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
if (key->flags) {
- I915_WRITE(DVSKEYVAL(pipe), key->min_value);
- I915_WRITE(DVSKEYMAX(pipe), key->max_value);
- I915_WRITE(DVSKEYMSK(pipe), key->channel_mask);
+ I915_WRITE_FW(DVSKEYVAL(pipe), key->min_value);
+ I915_WRITE_FW(DVSKEYMAX(pipe), key->max_value);
+ I915_WRITE_FW(DVSKEYMSK(pipe), key->channel_mask);
}
- if (key->flags & I915_SET_COLORKEY_DESTINATION)
- dvscntr |= DVS_DEST_KEY;
- else if (key->flags & I915_SET_COLORKEY_SOURCE)
- dvscntr |= DVS_SOURCE_KEY;
-
- I915_WRITE(DVSSTRIDE(pipe), fb->pitches[0]);
- I915_WRITE(DVSPOS(pipe), (crtc_y << 16) | crtc_x);
+ I915_WRITE_FW(DVSSTRIDE(pipe), fb->pitches[0]);
+ I915_WRITE_FW(DVSPOS(pipe), (crtc_y << 16) | crtc_x);
if (fb->modifier == I915_FORMAT_MOD_X_TILED)
- I915_WRITE(DVSTILEOFF(pipe), (y << 16) | x);
+ I915_WRITE_FW(DVSTILEOFF(pipe), (y << 16) | x);
else
- I915_WRITE(DVSLINOFF(pipe), linear_offset);
-
- I915_WRITE(DVSSIZE(pipe), (crtc_h << 16) | crtc_w);
- I915_WRITE(DVSSCALE(pipe), dvsscale);
- I915_WRITE(DVSCNTR(pipe), dvscntr);
- I915_WRITE(DVSSURF(pipe),
- intel_plane_ggtt_offset(plane_state) + dvssurf_offset);
- POSTING_READ(DVSSURF(pipe));
+ I915_WRITE_FW(DVSLINOFF(pipe), linear_offset);
+
+ I915_WRITE_FW(DVSSIZE(pipe), (crtc_h << 16) | crtc_w);
+ I915_WRITE_FW(DVSSCALE(pipe), dvsscale);
+ I915_WRITE_FW(DVSCNTR(pipe), dvscntr);
+ I915_WRITE_FW(DVSSURF(pipe),
+ intel_plane_ggtt_offset(plane_state) + dvssurf_offset);
+ POSTING_READ_FW(DVSSURF(pipe));
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static void
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_plane *intel_plane = to_intel_plane(plane);
int pipe = intel_plane->pipe;
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
- I915_WRITE(DVSCNTR(pipe), 0);
+ I915_WRITE_FW(DVSCNTR(pipe), 0);
/* Disable the scaler */
- I915_WRITE(DVSSCALE(pipe), 0);
+ I915_WRITE_FW(DVSSCALE(pipe), 0);
+
+ I915_WRITE_FW(DVSSURF(pipe), 0);
+ POSTING_READ_FW(DVSSURF(pipe));
- I915_WRITE(DVSSURF(pipe), 0);
- POSTING_READ(DVSSURF(pipe));
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
static int
intel_connector_attach_encoder(intel_connector, intel_encoder);
intel_encoder->type = INTEL_OUTPUT_TVOUT;
+ intel_encoder->power_domain = POWER_DOMAIN_PORT_OTHER;
intel_encoder->port = PORT_NONE;
intel_encoder->crtc_mask = (1 << 0) | (1 << 1);
intel_encoder->cloneable = 0;
#include "i915_drv.h"
#include "intel_uc.h"
+#include <linux/firmware.h>
+
+/* Reset GuC providing us with fresh state for both GuC and HuC.
+ */
+static int __intel_uc_reset_hw(struct drm_i915_private *dev_priv)
+{
+ int ret;
+ u32 guc_status;
+
+ ret = intel_guc_reset(dev_priv);
+ if (ret) {
+ DRM_ERROR("GuC reset failed, ret = %d\n", ret);
+ return ret;
+ }
+
+ guc_status = I915_READ(GUC_STATUS);
+ WARN(!(guc_status & GS_MIA_IN_RESET),
+ "GuC status: 0x%x, MIA core expected to be in reset\n",
+ guc_status);
+
+ return ret;
+}
+
+void intel_uc_sanitize_options(struct drm_i915_private *dev_priv)
+{
+ if (!HAS_GUC(dev_priv)) {
+ if (i915.enable_guc_loading > 0 ||
+ i915.enable_guc_submission > 0)
+ DRM_INFO("Ignoring GuC options, no hardware\n");
+
+ i915.enable_guc_loading = 0;
+ i915.enable_guc_submission = 0;
+ return;
+ }
+
+ /* A negative value means "use platform default" */
+ if (i915.enable_guc_loading < 0)
+ i915.enable_guc_loading = HAS_GUC_UCODE(dev_priv);
+
+ /* Verify firmware version */
+ if (i915.enable_guc_loading) {
+ if (HAS_HUC_UCODE(dev_priv))
+ intel_huc_select_fw(&dev_priv->huc);
+
+ if (intel_guc_select_fw(&dev_priv->guc))
+ i915.enable_guc_loading = 0;
+ }
+
+ /* Can't enable guc submission without guc loaded */
+ if (!i915.enable_guc_loading)
+ i915.enable_guc_submission = 0;
+
+ /* A negative value means "use platform default" */
+ if (i915.enable_guc_submission < 0)
+ i915.enable_guc_submission = HAS_GUC_SCHED(dev_priv);
+}
void intel_uc_init_early(struct drm_i915_private *dev_priv)
{
mutex_init(&dev_priv->guc.send_mutex);
}
+void intel_uc_init_fw(struct drm_i915_private *dev_priv)
+{
+ if (dev_priv->huc.fw.path)
+ intel_uc_prepare_fw(dev_priv, &dev_priv->huc.fw);
+
+ if (dev_priv->guc.fw.path)
+ intel_uc_prepare_fw(dev_priv, &dev_priv->guc.fw);
+}
+
+int intel_uc_init_hw(struct drm_i915_private *dev_priv)
+{
+ int ret, attempts;
+
+ /* GuC not enabled, nothing to do */
+ if (!i915.enable_guc_loading)
+ return 0;
+
+ gen9_reset_guc_interrupts(dev_priv);
+
+ /* We need to notify the guc whenever we change the GGTT */
+ i915_ggtt_enable_guc(dev_priv);
+
+ if (i915.enable_guc_submission) {
+ ret = i915_guc_submission_init(dev_priv);
+ if (ret)
+ goto err;
+ }
+
+ /* WaEnableuKernelHeaderValidFix:skl */
+ /* WaEnableGuCBootHashCheckNotSet:skl,bxt,kbl */
+ if (IS_GEN9(dev_priv))
+ attempts = 3;
+ else
+ attempts = 1;
+
+ while (attempts--) {
+ /*
+ * Always reset the GuC just before (re)loading, so
+ * that the state and timing are fairly predictable
+ */
+ ret = __intel_uc_reset_hw(dev_priv);
+ if (ret)
+ goto err_submission;
+
+ intel_huc_init_hw(&dev_priv->huc);
+ ret = intel_guc_init_hw(&dev_priv->guc);
+ if (ret == 0 || ret != -EAGAIN)
+ break;
+
+ DRM_DEBUG_DRIVER("GuC fw load failed: %d; will reset and "
+ "retry %d more time(s)\n", ret, attempts);
+ }
+
+ /* Did we succeded or run out of retries? */
+ if (ret)
+ goto err_submission;
+
+ intel_guc_auth_huc(dev_priv);
+ if (i915.enable_guc_submission) {
+ if (i915.guc_log_level >= 0)
+ gen9_enable_guc_interrupts(dev_priv);
+
+ ret = i915_guc_submission_enable(dev_priv);
+ if (ret)
+ goto err_submission;
+ }
+
+ return 0;
+
+ /*
+ * We've failed to load the firmware :(
+ *
+ * Decide whether to disable GuC submission and fall back to
+ * execlist mode, and whether to hide the error by returning
+ * zero or to return -EIO, which the caller will treat as a
+ * nonfatal error (i.e. it doesn't prevent driver load, but
+ * marks the GPU as wedged until reset).
+ */
+err_submission:
+ if (i915.enable_guc_submission)
+ i915_guc_submission_fini(dev_priv);
+
+err:
+ i915_ggtt_disable_guc(dev_priv);
+
+ DRM_ERROR("GuC init failed\n");
+ if (i915.enable_guc_loading > 1 || i915.enable_guc_submission > 1)
+ ret = -EIO;
+ else
+ ret = 0;
+
+ if (i915.enable_guc_submission) {
+ i915.enable_guc_submission = 0;
+ DRM_NOTE("Falling back from GuC submission to execlist mode\n");
+ }
+
+ return ret;
+}
+
/*
* Read GuC command/status register (SOFT_SCRATCH_0)
* Return true if it contains a response rather than a command
return intel_guc_send(guc, action, ARRAY_SIZE(action));
}
+void intel_uc_prepare_fw(struct drm_i915_private *dev_priv,
+ struct intel_uc_fw *uc_fw)
+{
+ struct pci_dev *pdev = dev_priv->drm.pdev;
+ struct drm_i915_gem_object *obj;
+ const struct firmware *fw = NULL;
+ struct uc_css_header *css;
+ size_t size;
+ int err;
+
+ uc_fw->fetch_status = INTEL_UC_FIRMWARE_PENDING;
+
+ DRM_DEBUG_DRIVER("before requesting firmware: uC fw fetch status %s\n",
+ intel_uc_fw_status_repr(uc_fw->fetch_status));
+
+ err = request_firmware(&fw, uc_fw->path, &pdev->dev);
+ if (err)
+ goto fail;
+ if (!fw)
+ goto fail;
+
+ DRM_DEBUG_DRIVER("fetch uC fw from %s succeeded, fw %p\n",
+ uc_fw->path, fw);
+
+ /* Check the size of the blob before examining buffer contents */
+ if (fw->size < sizeof(struct uc_css_header)) {
+ DRM_NOTE("Firmware header is missing\n");
+ goto fail;
+ }
+
+ css = (struct uc_css_header *)fw->data;
+
+ /* Firmware bits always start from header */
+ uc_fw->header_offset = 0;
+ uc_fw->header_size = (css->header_size_dw - css->modulus_size_dw -
+ css->key_size_dw - css->exponent_size_dw) * sizeof(u32);
+
+ if (uc_fw->header_size != sizeof(struct uc_css_header)) {
+ DRM_NOTE("CSS header definition mismatch\n");
+ goto fail;
+ }
+
+ /* then, uCode */
+ uc_fw->ucode_offset = uc_fw->header_offset + uc_fw->header_size;
+ uc_fw->ucode_size = (css->size_dw - css->header_size_dw) * sizeof(u32);
+
+ /* now RSA */
+ if (css->key_size_dw != UOS_RSA_SCRATCH_MAX_COUNT) {
+ DRM_NOTE("RSA key size is bad\n");
+ goto fail;
+ }
+ uc_fw->rsa_offset = uc_fw->ucode_offset + uc_fw->ucode_size;
+ uc_fw->rsa_size = css->key_size_dw * sizeof(u32);
+
+ /* At least, it should have header, uCode and RSA. Size of all three. */
+ size = uc_fw->header_size + uc_fw->ucode_size + uc_fw->rsa_size;
+ if (fw->size < size) {
+ DRM_NOTE("Missing firmware components\n");
+ goto fail;
+ }
+
+ /*
+ * The GuC firmware image has the version number embedded at a
+ * well-known offset within the firmware blob; note that major / minor
+ * version are TWO bytes each (i.e. u16), although all pointers and
+ * offsets are defined in terms of bytes (u8).
+ */
+ switch (uc_fw->type) {
+ case INTEL_UC_FW_TYPE_GUC:
+ /* Header and uCode will be loaded to WOPCM. Size of the two. */
+ size = uc_fw->header_size + uc_fw->ucode_size;
+
+ /* Top 32k of WOPCM is reserved (8K stack + 24k RC6 context). */
+ if (size > intel_guc_wopcm_size(dev_priv)) {
+ DRM_ERROR("Firmware is too large to fit in WOPCM\n");
+ goto fail;
+ }
+ uc_fw->major_ver_found = css->guc.sw_version >> 16;
+ uc_fw->minor_ver_found = css->guc.sw_version & 0xFFFF;
+ break;
+
+ case INTEL_UC_FW_TYPE_HUC:
+ uc_fw->major_ver_found = css->huc.sw_version >> 16;
+ uc_fw->minor_ver_found = css->huc.sw_version & 0xFFFF;
+ break;
+
+ default:
+ DRM_ERROR("Unknown firmware type %d\n", uc_fw->type);
+ err = -ENOEXEC;
+ goto fail;
+ }
+
+ if (uc_fw->major_ver_wanted == 0 && uc_fw->minor_ver_wanted == 0) {
+ DRM_NOTE("Skipping uC firmware version check\n");
+ } else if (uc_fw->major_ver_found != uc_fw->major_ver_wanted ||
+ uc_fw->minor_ver_found < uc_fw->minor_ver_wanted) {
+ DRM_NOTE("uC firmware version %d.%d, required %d.%d\n",
+ uc_fw->major_ver_found, uc_fw->minor_ver_found,
+ uc_fw->major_ver_wanted, uc_fw->minor_ver_wanted);
+ err = -ENOEXEC;
+ goto fail;
+ }
+
+ DRM_DEBUG_DRIVER("firmware version %d.%d OK (minimum %d.%d)\n",
+ uc_fw->major_ver_found, uc_fw->minor_ver_found,
+ uc_fw->major_ver_wanted, uc_fw->minor_ver_wanted);
+
+ obj = i915_gem_object_create_from_data(dev_priv, fw->data, fw->size);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto fail;
+ }
+
+ uc_fw->obj = obj;
+ uc_fw->size = fw->size;
+
+ DRM_DEBUG_DRIVER("uC fw fetch status SUCCESS, obj %p\n",
+ uc_fw->obj);
+
+ release_firmware(fw);
+ uc_fw->fetch_status = INTEL_UC_FIRMWARE_SUCCESS;
+ return;
+
+fail:
+ DRM_WARN("Failed to fetch valid uC firmware from %s (error %d)\n",
+ uc_fw->path, err);
+ DRM_DEBUG_DRIVER("uC fw fetch status FAIL; err %d, fw %p, obj %p\n",
+ err, fw, uc_fw->obj);
+
+ release_firmware(fw); /* OK even if fw is NULL */
+ uc_fw->fetch_status = INTEL_UC_FIRMWARE_FAIL;
+}
uint16_t major_ver_found;
uint16_t minor_ver_found;
- enum intel_uc_fw_type fw;
+ enum intel_uc_fw_type type;
uint32_t header_size;
uint32_t header_offset;
uint32_t rsa_size;
};
/* intel_uc.c */
+void intel_uc_sanitize_options(struct drm_i915_private *dev_priv);
void intel_uc_init_early(struct drm_i915_private *dev_priv);
+void intel_uc_init_fw(struct drm_i915_private *dev_priv);
+int intel_uc_init_hw(struct drm_i915_private *dev_priv);
+void intel_uc_prepare_fw(struct drm_i915_private *dev_priv,
+ struct intel_uc_fw *uc_fw);
int intel_guc_send(struct intel_guc *guc, const u32 *action, u32 len);
int intel_guc_sample_forcewake(struct intel_guc *guc);
/* intel_guc_loader.c */
-extern void intel_guc_init(struct drm_i915_private *dev_priv);
-extern int intel_guc_setup(struct drm_i915_private *dev_priv);
-extern void intel_guc_fini(struct drm_i915_private *dev_priv);
-extern const char *intel_uc_fw_status_repr(enum intel_uc_fw_status status);
-extern int intel_guc_suspend(struct drm_i915_private *dev_priv);
-extern int intel_guc_resume(struct drm_i915_private *dev_priv);
-void intel_uc_fw_fetch(struct drm_i915_private *dev_priv,
- struct intel_uc_fw *uc_fw);
+int intel_guc_select_fw(struct intel_guc *guc);
+int intel_guc_init_hw(struct intel_guc *guc);
+void intel_guc_fini(struct drm_i915_private *dev_priv);
+const char *intel_uc_fw_status_repr(enum intel_uc_fw_status status);
+int intel_guc_suspend(struct drm_i915_private *dev_priv);
+int intel_guc_resume(struct drm_i915_private *dev_priv);
u32 intel_guc_wopcm_size(struct drm_i915_private *dev_priv);
/* i915_guc_submission.c */
}
/* intel_huc.c */
-void intel_huc_init(struct drm_i915_private *dev_priv);
+void intel_huc_select_fw(struct intel_huc *huc);
void intel_huc_fini(struct drm_i915_private *dev_priv);
-int intel_huc_load(struct drm_i915_private *dev_priv);
+int intel_huc_init_hw(struct intel_huc *huc);
void intel_guc_auth_huc(struct drm_i915_private *dev_priv);
#endif
#include "intel_drv.h"
#include "i915_vgpu.h"
+#include <asm/iosf_mbi.h>
#include <linux/pm_runtime.h>
#define FORCEWAKE_ACK_TIMEOUT_MS 50
return HRTIMER_NORESTART;
}
-void intel_uncore_forcewake_reset(struct drm_i915_private *dev_priv,
- bool restore)
+static void intel_uncore_forcewake_reset(struct drm_i915_private *dev_priv,
+ bool restore)
{
unsigned long irqflags;
struct intel_uncore_forcewake_domain *domain;
intel_uncore_forcewake_reset(dev_priv, restore_forcewake);
}
-void intel_uncore_early_sanitize(struct drm_i915_private *dev_priv,
- bool restore_forcewake)
+void intel_uncore_suspend(struct drm_i915_private *dev_priv)
{
- __intel_uncore_early_sanitize(dev_priv, restore_forcewake);
+ iosf_mbi_unregister_pmic_bus_access_notifier(
+ &dev_priv->uncore.pmic_bus_access_nb);
+ intel_uncore_forcewake_reset(dev_priv, false);
+}
+
+void intel_uncore_resume_early(struct drm_i915_private *dev_priv)
+{
+ __intel_uncore_early_sanitize(dev_priv, true);
+ iosf_mbi_register_pmic_bus_access_notifier(
+ &dev_priv->uncore.pmic_bus_access_nb);
i915_check_and_clear_faults(dev_priv);
}
void intel_uncore_forcewake_get__locked(struct drm_i915_private *dev_priv,
enum forcewake_domains fw_domains)
{
- assert_spin_locked(&dev_priv->uncore.lock);
+ lockdep_assert_held(&dev_priv->uncore.lock);
if (!dev_priv->uncore.funcs.force_wake_get)
return;
void intel_uncore_forcewake_put__locked(struct drm_i915_private *dev_priv,
enum forcewake_domains fw_domains)
{
- assert_spin_locked(&dev_priv->uncore.lock);
+ lockdep_assert_held(&dev_priv->uncore.lock);
if (!dev_priv->uncore.funcs.force_wake_put)
return;
return entry->domains;
}
-static void
-intel_fw_table_check(struct drm_i915_private *dev_priv)
-{
- const struct intel_forcewake_range *ranges;
- unsigned int num_ranges;
- s32 prev;
- unsigned int i;
-
- if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG))
- return;
-
- ranges = dev_priv->uncore.fw_domains_table;
- if (!ranges)
- return;
-
- num_ranges = dev_priv->uncore.fw_domains_table_entries;
-
- for (i = 0, prev = -1; i < num_ranges; i++, ranges++) {
- WARN_ON_ONCE(IS_GEN9(dev_priv) &&
- (prev + 1) != (s32)ranges->start);
- WARN_ON_ONCE(prev >= (s32)ranges->start);
- prev = ranges->start;
- WARN_ON_ONCE(prev >= (s32)ranges->end);
- prev = ranges->end;
- }
-}
-
#define GEN_FW_RANGE(s, e, d) \
{ .start = (s), .end = (e), .domains = (d) }
/* TODO: Other registers are not yet used */
};
-static void intel_shadow_table_check(void)
-{
- const i915_reg_t *reg = gen8_shadowed_regs;
- s32 prev;
- u32 offset;
- unsigned int i;
-
- if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG))
- return;
-
- for (i = 0, prev = -1; i < ARRAY_SIZE(gen8_shadowed_regs); i++, reg++) {
- offset = i915_mmio_reg_offset(*reg);
- WARN_ON_ONCE(prev >= (s32)offset);
- prev = offset;
- }
-}
-
static int mmio_reg_cmp(u32 key, const i915_reg_t *reg)
{
u32 offset = i915_mmio_reg_offset(*reg);
___force_wake_auto(dev_priv, fw_domains);
}
-#define __gen6_read(x) \
-static u##x \
-gen6_read##x(struct drm_i915_private *dev_priv, i915_reg_t reg, bool trace) { \
- enum forcewake_domains fw_engine; \
- GEN6_READ_HEADER(x); \
- fw_engine = __gen6_reg_read_fw_domains(offset); \
- if (fw_engine) \
- __force_wake_auto(dev_priv, fw_engine); \
- val = __raw_i915_read##x(dev_priv, reg); \
- GEN6_READ_FOOTER; \
-}
-
-#define __fwtable_read(x) \
+#define __gen_read(func, x) \
static u##x \
-fwtable_read##x(struct drm_i915_private *dev_priv, i915_reg_t reg, bool trace) { \
+func##_read##x(struct drm_i915_private *dev_priv, i915_reg_t reg, bool trace) { \
enum forcewake_domains fw_engine; \
GEN6_READ_HEADER(x); \
- fw_engine = __fwtable_reg_read_fw_domains(offset); \
+ fw_engine = __##func##_reg_read_fw_domains(offset); \
if (fw_engine) \
__force_wake_auto(dev_priv, fw_engine); \
val = __raw_i915_read##x(dev_priv, reg); \
GEN6_READ_FOOTER; \
}
+#define __gen6_read(x) __gen_read(gen6, x)
+#define __fwtable_read(x) __gen_read(fwtable, x)
#define __gen9_decoupled_read(x) \
static u##x \
#undef GEN6_READ_FOOTER
#undef GEN6_READ_HEADER
-#define VGPU_READ_HEADER(x) \
- unsigned long irqflags; \
- u##x val = 0; \
- assert_rpm_device_not_suspended(dev_priv); \
- spin_lock_irqsave(&dev_priv->uncore.lock, irqflags)
-
-#define VGPU_READ_FOOTER \
- spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags); \
- trace_i915_reg_rw(false, reg, val, sizeof(val), trace); \
- return val
-
-#define __vgpu_read(x) \
-static u##x \
-vgpu_read##x(struct drm_i915_private *dev_priv, i915_reg_t reg, bool trace) { \
- VGPU_READ_HEADER(x); \
- val = __raw_i915_read##x(dev_priv, reg); \
- VGPU_READ_FOOTER; \
-}
-
-__vgpu_read(8)
-__vgpu_read(16)
-__vgpu_read(32)
-__vgpu_read(64)
-
-#undef __vgpu_read
-#undef VGPU_READ_FOOTER
-#undef VGPU_READ_HEADER
-
#define GEN2_WRITE_HEADER \
trace_i915_reg_rw(true, reg, val, sizeof(val), trace); \
assert_rpm_wakelock_held(dev_priv); \
GEN6_WRITE_FOOTER; \
}
-#define __gen8_write(x) \
+#define __gen_write(func, x) \
static void \
-gen8_write##x(struct drm_i915_private *dev_priv, i915_reg_t reg, u##x val, bool trace) { \
+func##_write##x(struct drm_i915_private *dev_priv, i915_reg_t reg, u##x val, bool trace) { \
enum forcewake_domains fw_engine; \
GEN6_WRITE_HEADER; \
- fw_engine = __gen8_reg_write_fw_domains(offset); \
- if (fw_engine) \
- __force_wake_auto(dev_priv, fw_engine); \
- __raw_i915_write##x(dev_priv, reg, val); \
- GEN6_WRITE_FOOTER; \
-}
-
-#define __fwtable_write(x) \
-static void \
-fwtable_write##x(struct drm_i915_private *dev_priv, i915_reg_t reg, u##x val, bool trace) { \
- enum forcewake_domains fw_engine; \
- GEN6_WRITE_HEADER; \
- fw_engine = __fwtable_reg_write_fw_domains(offset); \
+ fw_engine = __##func##_reg_write_fw_domains(offset); \
if (fw_engine) \
__force_wake_auto(dev_priv, fw_engine); \
__raw_i915_write##x(dev_priv, reg, val); \
GEN6_WRITE_FOOTER; \
}
+#define __gen8_write(x) __gen_write(gen8, x)
+#define __fwtable_write(x) __gen_write(fwtable, x)
#define __gen9_decoupled_write(x) \
static void \
#undef GEN6_WRITE_FOOTER
#undef GEN6_WRITE_HEADER
-#define VGPU_WRITE_HEADER \
- unsigned long irqflags; \
- trace_i915_reg_rw(true, reg, val, sizeof(val), trace); \
- assert_rpm_device_not_suspended(dev_priv); \
- spin_lock_irqsave(&dev_priv->uncore.lock, irqflags)
-
-#define VGPU_WRITE_FOOTER \
- spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags)
-
-#define __vgpu_write(x) \
-static void vgpu_write##x(struct drm_i915_private *dev_priv, \
- i915_reg_t reg, u##x val, bool trace) { \
- VGPU_WRITE_HEADER; \
- __raw_i915_write##x(dev_priv, reg, val); \
- VGPU_WRITE_FOOTER; \
-}
-
-__vgpu_write(8)
-__vgpu_write(16)
-__vgpu_write(32)
-
-#undef __vgpu_write
-#undef VGPU_WRITE_FOOTER
-#undef VGPU_WRITE_HEADER
-
#define ASSIGN_WRITE_MMIO_VFUNCS(x) \
do { \
dev_priv->uncore.funcs.mmio_writeb = x##_write8; \
static void intel_uncore_fw_domains_init(struct drm_i915_private *dev_priv)
{
- if (INTEL_INFO(dev_priv)->gen <= 5)
+ if (INTEL_GEN(dev_priv) <= 5 || intel_vgpu_active(dev_priv))
return;
if (IS_GEN9(dev_priv)) {
dev_priv->uncore.fw_domains_table_entries = ARRAY_SIZE((d)); \
}
+static int i915_pmic_bus_access_notifier(struct notifier_block *nb,
+ unsigned long action, void *data)
+{
+ struct drm_i915_private *dev_priv = container_of(nb,
+ struct drm_i915_private, uncore.pmic_bus_access_nb);
+
+ switch (action) {
+ case MBI_PMIC_BUS_ACCESS_BEGIN:
+ /*
+ * forcewake all now to make sure that we don't need to do a
+ * forcewake later which on systems where this notifier gets
+ * called requires the punit to access to the shared pmic i2c
+ * bus, which will be busy after this notification, leading to:
+ * "render: timed out waiting for forcewake ack request."
+ * errors.
+ */
+ intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+ break;
+ case MBI_PMIC_BUS_ACCESS_END:
+ intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
void intel_uncore_init(struct drm_i915_private *dev_priv)
{
i915_check_vgpu(dev_priv);
__intel_uncore_early_sanitize(dev_priv, false);
dev_priv->uncore.unclaimed_mmio_check = 1;
+ dev_priv->uncore.pmic_bus_access_nb.notifier_call =
+ i915_pmic_bus_access_notifier;
- switch (INTEL_INFO(dev_priv)->gen) {
- default:
- case 9:
- ASSIGN_FW_DOMAINS_TABLE(__gen9_fw_ranges);
- ASSIGN_WRITE_MMIO_VFUNCS(fwtable);
- ASSIGN_READ_MMIO_VFUNCS(fwtable);
- if (HAS_DECOUPLED_MMIO(dev_priv)) {
- dev_priv->uncore.funcs.mmio_readl =
- gen9_decoupled_read32;
- dev_priv->uncore.funcs.mmio_readq =
- gen9_decoupled_read64;
- dev_priv->uncore.funcs.mmio_writel =
- gen9_decoupled_write32;
+ if (IS_GEN(dev_priv, 2, 4) || intel_vgpu_active(dev_priv)) {
+ ASSIGN_WRITE_MMIO_VFUNCS(gen2);
+ ASSIGN_READ_MMIO_VFUNCS(gen2);
+ } else if (IS_GEN5(dev_priv)) {
+ ASSIGN_WRITE_MMIO_VFUNCS(gen5);
+ ASSIGN_READ_MMIO_VFUNCS(gen5);
+ } else if (IS_GEN(dev_priv, 6, 7)) {
+ ASSIGN_WRITE_MMIO_VFUNCS(gen6);
+
+ if (IS_VALLEYVIEW(dev_priv)) {
+ ASSIGN_FW_DOMAINS_TABLE(__vlv_fw_ranges);
+ ASSIGN_READ_MMIO_VFUNCS(fwtable);
+ } else {
+ ASSIGN_READ_MMIO_VFUNCS(gen6);
}
- break;
- case 8:
+ } else if (IS_GEN8(dev_priv)) {
if (IS_CHERRYVIEW(dev_priv)) {
ASSIGN_FW_DOMAINS_TABLE(__chv_fw_ranges);
ASSIGN_WRITE_MMIO_VFUNCS(fwtable);
ASSIGN_WRITE_MMIO_VFUNCS(gen8);
ASSIGN_READ_MMIO_VFUNCS(gen6);
}
- break;
- case 7:
- case 6:
- ASSIGN_WRITE_MMIO_VFUNCS(gen6);
-
- if (IS_VALLEYVIEW(dev_priv)) {
- ASSIGN_FW_DOMAINS_TABLE(__vlv_fw_ranges);
- ASSIGN_READ_MMIO_VFUNCS(fwtable);
- } else {
- ASSIGN_READ_MMIO_VFUNCS(gen6);
+ } else {
+ ASSIGN_FW_DOMAINS_TABLE(__gen9_fw_ranges);
+ ASSIGN_WRITE_MMIO_VFUNCS(fwtable);
+ ASSIGN_READ_MMIO_VFUNCS(fwtable);
+ if (HAS_DECOUPLED_MMIO(dev_priv)) {
+ dev_priv->uncore.funcs.mmio_readl =
+ gen9_decoupled_read32;
+ dev_priv->uncore.funcs.mmio_readq =
+ gen9_decoupled_read64;
+ dev_priv->uncore.funcs.mmio_writel =
+ gen9_decoupled_write32;
}
- break;
- case 5:
- ASSIGN_WRITE_MMIO_VFUNCS(gen5);
- ASSIGN_READ_MMIO_VFUNCS(gen5);
- break;
- case 4:
- case 3:
- case 2:
- ASSIGN_WRITE_MMIO_VFUNCS(gen2);
- ASSIGN_READ_MMIO_VFUNCS(gen2);
- break;
}
- intel_fw_table_check(dev_priv);
- if (INTEL_GEN(dev_priv) >= 8)
- intel_shadow_table_check();
-
- if (intel_vgpu_active(dev_priv)) {
- ASSIGN_WRITE_MMIO_VFUNCS(vgpu);
- ASSIGN_READ_MMIO_VFUNCS(vgpu);
- }
+ iosf_mbi_register_pmic_bus_access_notifier(
+ &dev_priv->uncore.pmic_bus_access_nb);
i915_check_and_clear_faults(dev_priv);
}
void intel_uncore_fini(struct drm_i915_private *dev_priv)
{
+ iosf_mbi_unregister_pmic_bus_access_notifier(
+ &dev_priv->uncore.pmic_bus_access_nb);
+
/* Paranoia: make sure we have disabled everything before we exit. */
intel_uncore_sanitize(dev_priv);
intel_uncore_forcewake_reset(dev_priv, false);
return fw_domains;
}
+
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+#include "selftests/intel_uncore.c"
+#endif
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "huge_gem_object.h"
+
+static void huge_free_pages(struct drm_i915_gem_object *obj,
+ struct sg_table *pages)
+{
+ unsigned long nreal = obj->scratch / PAGE_SIZE;
+ struct scatterlist *sg;
+
+ for (sg = pages->sgl; sg && nreal--; sg = __sg_next(sg))
+ __free_page(sg_page(sg));
+
+ sg_free_table(pages);
+ kfree(pages);
+}
+
+static struct sg_table *
+huge_get_pages(struct drm_i915_gem_object *obj)
+{
+#define GFP (GFP_KERNEL | __GFP_NOWARN | __GFP_NORETRY)
+ const unsigned long nreal = obj->scratch / PAGE_SIZE;
+ const unsigned long npages = obj->base.size / PAGE_SIZE;
+ struct scatterlist *sg, *src, *end;
+ struct sg_table *pages;
+ unsigned long n;
+
+ pages = kmalloc(sizeof(*pages), GFP);
+ if (!pages)
+ return ERR_PTR(-ENOMEM);
+
+ if (sg_alloc_table(pages, npages, GFP)) {
+ kfree(pages);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ sg = pages->sgl;
+ for (n = 0; n < nreal; n++) {
+ struct page *page;
+
+ page = alloc_page(GFP | __GFP_HIGHMEM);
+ if (!page) {
+ sg_mark_end(sg);
+ goto err;
+ }
+
+ sg_set_page(sg, page, PAGE_SIZE, 0);
+ sg = __sg_next(sg);
+ }
+ if (nreal < npages) {
+ for (end = sg, src = pages->sgl; sg; sg = __sg_next(sg)) {
+ sg_set_page(sg, sg_page(src), PAGE_SIZE, 0);
+ src = __sg_next(src);
+ if (src == end)
+ src = pages->sgl;
+ }
+ }
+
+ if (i915_gem_gtt_prepare_pages(obj, pages))
+ goto err;
+
+ return pages;
+
+err:
+ huge_free_pages(obj, pages);
+ return ERR_PTR(-ENOMEM);
+#undef GFP
+}
+
+static void huge_put_pages(struct drm_i915_gem_object *obj,
+ struct sg_table *pages)
+{
+ i915_gem_gtt_finish_pages(obj, pages);
+ huge_free_pages(obj, pages);
+
+ obj->mm.dirty = false;
+}
+
+static const struct drm_i915_gem_object_ops huge_ops = {
+ .flags = I915_GEM_OBJECT_HAS_STRUCT_PAGE |
+ I915_GEM_OBJECT_IS_SHRINKABLE,
+ .get_pages = huge_get_pages,
+ .put_pages = huge_put_pages,
+};
+
+struct drm_i915_gem_object *
+huge_gem_object(struct drm_i915_private *i915,
+ phys_addr_t phys_size,
+ dma_addr_t dma_size)
+{
+ struct drm_i915_gem_object *obj;
+
+ GEM_BUG_ON(!phys_size || phys_size > dma_size);
+ GEM_BUG_ON(!IS_ALIGNED(phys_size, PAGE_SIZE));
+ GEM_BUG_ON(!IS_ALIGNED(dma_size, I915_GTT_PAGE_SIZE));
+
+ if (overflows_type(dma_size, obj->base.size))
+ return ERR_PTR(-E2BIG);
+
+ obj = i915_gem_object_alloc(i915);
+ if (!obj)
+ return ERR_PTR(-ENOMEM);
+
+ drm_gem_private_object_init(&i915->drm, &obj->base, dma_size);
+ i915_gem_object_init(obj, &huge_ops);
+
+ obj->base.write_domain = I915_GEM_DOMAIN_CPU;
+ obj->base.read_domains = I915_GEM_DOMAIN_CPU;
+ obj->cache_level = HAS_LLC(i915) ? I915_CACHE_LLC : I915_CACHE_NONE;
+ obj->scratch = phys_size;
+
+ return obj;
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __HUGE_GEM_OBJECT_H
+#define __HUGE_GEM_OBJECT_H
+
+struct drm_i915_gem_object *
+huge_gem_object(struct drm_i915_private *i915,
+ phys_addr_t phys_size,
+ dma_addr_t dma_size);
+
+static inline phys_addr_t
+huge_gem_object_phys_size(struct drm_i915_gem_object *obj)
+{
+ return obj->scratch;
+}
+
+static inline dma_addr_t
+huge_gem_object_dma_size(struct drm_i915_gem_object *obj)
+{
+ return obj->base.size;
+}
+
+#endif /* !__HUGE_GEM_OBJECT_H */
--- /dev/null
+/*
+ * Copyright © 2017 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include <linux/prime_numbers.h>
+
+#include "../i915_selftest.h"
+#include "i915_random.h"
+
+static int cpu_set(struct drm_i915_gem_object *obj,
+ unsigned long offset,
+ u32 v)
+{
+ unsigned int needs_clflush;
+ struct page *page;
+ typeof(v) *map;
+ int err;
+
+ err = i915_gem_obj_prepare_shmem_write(obj, &needs_clflush);
+ if (err)
+ return err;
+
+ page = i915_gem_object_get_page(obj, offset >> PAGE_SHIFT);
+ map = kmap_atomic(page);
+ if (needs_clflush & CLFLUSH_BEFORE)
+ clflush(map+offset_in_page(offset) / sizeof(*map));
+ map[offset_in_page(offset) / sizeof(*map)] = v;
+ if (needs_clflush & CLFLUSH_AFTER)
+ clflush(map+offset_in_page(offset) / sizeof(*map));
+ kunmap_atomic(map);
+
+ i915_gem_obj_finish_shmem_access(obj);
+ return 0;
+}
+
+static int cpu_get(struct drm_i915_gem_object *obj,
+ unsigned long offset,
+ u32 *v)
+{
+ unsigned int needs_clflush;
+ struct page *page;
+ typeof(v) map;
+ int err;
+
+ err = i915_gem_obj_prepare_shmem_read(obj, &needs_clflush);
+ if (err)
+ return err;
+
+ page = i915_gem_object_get_page(obj, offset >> PAGE_SHIFT);
+ map = kmap_atomic(page);
+ if (needs_clflush & CLFLUSH_BEFORE)
+ clflush(map+offset_in_page(offset) / sizeof(*map));
+ *v = map[offset_in_page(offset) / sizeof(*map)];
+ kunmap_atomic(map);
+
+ i915_gem_obj_finish_shmem_access(obj);
+ return 0;
+}
+
+static int gtt_set(struct drm_i915_gem_object *obj,
+ unsigned long offset,
+ u32 v)
+{
+ struct i915_vma *vma;
+ typeof(v) *map;
+ int err;
+
+ err = i915_gem_object_set_to_gtt_domain(obj, true);
+ if (err)
+ return err;
+
+ vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0, PIN_MAPPABLE);
+ if (IS_ERR(vma))
+ return PTR_ERR(vma);
+
+ map = i915_vma_pin_iomap(vma);
+ i915_vma_unpin(vma);
+ if (IS_ERR(map))
+ return PTR_ERR(map);
+
+ map[offset / sizeof(*map)] = v;
+ i915_vma_unpin_iomap(vma);
+
+ return 0;
+}
+
+static int gtt_get(struct drm_i915_gem_object *obj,
+ unsigned long offset,
+ u32 *v)
+{
+ struct i915_vma *vma;
+ typeof(v) map;
+ int err;
+
+ err = i915_gem_object_set_to_gtt_domain(obj, false);
+ if (err)
+ return err;
+
+ vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0, PIN_MAPPABLE);
+ if (IS_ERR(vma))
+ return PTR_ERR(vma);
+
+ map = i915_vma_pin_iomap(vma);
+ i915_vma_unpin(vma);
+ if (IS_ERR(map))
+ return PTR_ERR(map);
+
+ *v = map[offset / sizeof(*map)];
+ i915_vma_unpin_iomap(vma);
+
+ return 0;
+}
+
+static int wc_set(struct drm_i915_gem_object *obj,
+ unsigned long offset,
+ u32 v)
+{
+ typeof(v) *map;
+ int err;
+
+ /* XXX GTT write followed by WC write go missing */
+ i915_gem_object_flush_gtt_write_domain(obj);
+
+ err = i915_gem_object_set_to_gtt_domain(obj, true);
+ if (err)
+ return err;
+
+ map = i915_gem_object_pin_map(obj, I915_MAP_WC);
+ if (IS_ERR(map))
+ return PTR_ERR(map);
+
+ map[offset / sizeof(*map)] = v;
+ i915_gem_object_unpin_map(obj);
+
+ return 0;
+}
+
+static int wc_get(struct drm_i915_gem_object *obj,
+ unsigned long offset,
+ u32 *v)
+{
+ typeof(v) map;
+ int err;
+
+ /* XXX WC write followed by GTT write go missing */
+ i915_gem_object_flush_gtt_write_domain(obj);
+
+ err = i915_gem_object_set_to_gtt_domain(obj, false);
+ if (err)
+ return err;
+
+ map = i915_gem_object_pin_map(obj, I915_MAP_WC);
+ if (IS_ERR(map))
+ return PTR_ERR(map);
+
+ *v = map[offset / sizeof(*map)];
+ i915_gem_object_unpin_map(obj);
+
+ return 0;
+}
+
+static int gpu_set(struct drm_i915_gem_object *obj,
+ unsigned long offset,
+ u32 v)
+{
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+ struct drm_i915_gem_request *rq;
+ struct i915_vma *vma;
+ u32 *cs;
+ int err;
+
+ err = i915_gem_object_set_to_gtt_domain(obj, true);
+ if (err)
+ return err;
+
+ vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0, 0);
+ if (IS_ERR(vma))
+ return PTR_ERR(vma);
+
+ rq = i915_gem_request_alloc(i915->engine[RCS], i915->kernel_context);
+ if (IS_ERR(rq)) {
+ i915_vma_unpin(vma);
+ return PTR_ERR(rq);
+ }
+
+ cs = intel_ring_begin(rq, 4);
+ if (IS_ERR(cs)) {
+ __i915_add_request(rq, false);
+ i915_vma_unpin(vma);
+ return PTR_ERR(cs);
+ }
+
+ if (INTEL_GEN(i915) >= 8) {
+ *cs++ = MI_STORE_DWORD_IMM_GEN4 | 1 << 22;
+ *cs++ = lower_32_bits(i915_ggtt_offset(vma) + offset);
+ *cs++ = upper_32_bits(i915_ggtt_offset(vma) + offset);
+ *cs++ = v;
+ } else if (INTEL_GEN(i915) >= 4) {
+ *cs++ = MI_STORE_DWORD_IMM_GEN4 | 1 << 22;
+ *cs++ = 0;
+ *cs++ = i915_ggtt_offset(vma) + offset;
+ *cs++ = v;
+ } else {
+ *cs++ = MI_STORE_DWORD_IMM | 1 << 22;
+ *cs++ = i915_ggtt_offset(vma) + offset;
+ *cs++ = v;
+ *cs++ = MI_NOOP;
+ }
+ intel_ring_advance(rq, cs);
+
+ i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
+ i915_vma_unpin(vma);
+
+ reservation_object_lock(obj->resv, NULL);
+ reservation_object_add_excl_fence(obj->resv, &rq->fence);
+ reservation_object_unlock(obj->resv);
+
+ __i915_add_request(rq, true);
+
+ return 0;
+}
+
+static bool always_valid(struct drm_i915_private *i915)
+{
+ return true;
+}
+
+static bool needs_mi_store_dword(struct drm_i915_private *i915)
+{
+ return igt_can_mi_store_dword_imm(i915);
+}
+
+static const struct igt_coherency_mode {
+ const char *name;
+ int (*set)(struct drm_i915_gem_object *, unsigned long offset, u32 v);
+ int (*get)(struct drm_i915_gem_object *, unsigned long offset, u32 *v);
+ bool (*valid)(struct drm_i915_private *i915);
+} igt_coherency_mode[] = {
+ { "cpu", cpu_set, cpu_get, always_valid },
+ { "gtt", gtt_set, gtt_get, always_valid },
+ { "wc", wc_set, wc_get, always_valid },
+ { "gpu", gpu_set, NULL, needs_mi_store_dword },
+ { },
+};
+
+static int igt_gem_coherency(void *arg)
+{
+ const unsigned int ncachelines = PAGE_SIZE/64;
+ I915_RND_STATE(prng);
+ struct drm_i915_private *i915 = arg;
+ const struct igt_coherency_mode *read, *write, *over;
+ struct drm_i915_gem_object *obj;
+ unsigned long count, n;
+ u32 *offsets, *values;
+ int err = 0;
+
+ /* We repeatedly write, overwrite and read from a sequence of
+ * cachelines in order to try and detect incoherency (unflushed writes
+ * from either the CPU or GPU). Each setter/getter uses our cache
+ * domain API which should prevent incoherency.
+ */
+
+ offsets = kmalloc_array(ncachelines, 2*sizeof(u32), GFP_KERNEL);
+ if (!offsets)
+ return -ENOMEM;
+ for (count = 0; count < ncachelines; count++)
+ offsets[count] = count * 64 + 4 * (count % 16);
+
+ values = offsets + ncachelines;
+
+ mutex_lock(&i915->drm.struct_mutex);
+ for (over = igt_coherency_mode; over->name; over++) {
+ if (!over->set)
+ continue;
+
+ if (!over->valid(i915))
+ continue;
+
+ for (write = igt_coherency_mode; write->name; write++) {
+ if (!write->set)
+ continue;
+
+ if (!write->valid(i915))
+ continue;
+
+ for (read = igt_coherency_mode; read->name; read++) {
+ if (!read->get)
+ continue;
+
+ if (!read->valid(i915))
+ continue;
+
+ for_each_prime_number_from(count, 1, ncachelines) {
+ obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto unlock;
+ }
+
+ i915_random_reorder(offsets, ncachelines, &prng);
+ for (n = 0; n < count; n++)
+ values[n] = prandom_u32_state(&prng);
+
+ for (n = 0; n < count; n++) {
+ err = over->set(obj, offsets[n], ~values[n]);
+ if (err) {
+ pr_err("Failed to set stale value[%ld/%ld] in object using %s, err=%d\n",
+ n, count, over->name, err);
+ goto put_object;
+ }
+ }
+
+ for (n = 0; n < count; n++) {
+ err = write->set(obj, offsets[n], values[n]);
+ if (err) {
+ pr_err("Failed to set value[%ld/%ld] in object using %s, err=%d\n",
+ n, count, write->name, err);
+ goto put_object;
+ }
+ }
+
+ for (n = 0; n < count; n++) {
+ u32 found;
+
+ err = read->get(obj, offsets[n], &found);
+ if (err) {
+ pr_err("Failed to get value[%ld/%ld] in object using %s, err=%d\n",
+ n, count, read->name, err);
+ goto put_object;
+ }
+
+ if (found != values[n]) {
+ pr_err("Value[%ld/%ld] mismatch, (overwrite with %s) wrote [%s] %x read [%s] %x (inverse %x), at offset %x\n",
+ n, count, over->name,
+ write->name, values[n],
+ read->name, found,
+ ~values[n], offsets[n]);
+ err = -EINVAL;
+ goto put_object;
+ }
+ }
+
+ __i915_gem_object_release_unless_active(obj);
+ }
+ }
+ }
+ }
+unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ kfree(offsets);
+ return err;
+
+put_object:
+ __i915_gem_object_release_unless_active(obj);
+ goto unlock;
+}
+
+int i915_gem_coherency_live_selftests(struct drm_i915_private *i915)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_gem_coherency),
+ };
+
+ return i915_subtests(tests, i915);
+}
--- /dev/null
+/*
+ * Copyright © 2017 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "../i915_selftest.h"
+
+#include "mock_drm.h"
+#include "huge_gem_object.h"
+
+#define DW_PER_PAGE (PAGE_SIZE / sizeof(u32))
+
+static struct i915_vma *
+gpu_fill_dw(struct i915_vma *vma, u64 offset, unsigned long count, u32 value)
+{
+ struct drm_i915_gem_object *obj;
+ const int gen = INTEL_GEN(vma->vm->i915);
+ unsigned long n, size;
+ u32 *cmd;
+ int err;
+
+ GEM_BUG_ON(!igt_can_mi_store_dword_imm(vma->vm->i915));
+
+ size = (4 * count + 1) * sizeof(u32);
+ size = round_up(size, PAGE_SIZE);
+ obj = i915_gem_object_create_internal(vma->vm->i915, size);
+ if (IS_ERR(obj))
+ return ERR_CAST(obj);
+
+ cmd = i915_gem_object_pin_map(obj, I915_MAP_WB);
+ if (IS_ERR(cmd)) {
+ err = PTR_ERR(cmd);
+ goto err;
+ }
+
+ GEM_BUG_ON(offset + (count - 1) * PAGE_SIZE > vma->node.size);
+ offset += vma->node.start;
+
+ for (n = 0; n < count; n++) {
+ if (gen >= 8) {
+ *cmd++ = MI_STORE_DWORD_IMM_GEN4;
+ *cmd++ = lower_32_bits(offset);
+ *cmd++ = upper_32_bits(offset);
+ *cmd++ = value;
+ } else if (gen >= 4) {
+ *cmd++ = MI_STORE_DWORD_IMM_GEN4 |
+ (gen < 6 ? 1 << 22 : 0);
+ *cmd++ = 0;
+ *cmd++ = offset;
+ *cmd++ = value;
+ } else {
+ *cmd++ = MI_STORE_DWORD_IMM | 1 << 22;
+ *cmd++ = offset;
+ *cmd++ = value;
+ }
+ offset += PAGE_SIZE;
+ }
+ *cmd = MI_BATCH_BUFFER_END;
+ i915_gem_object_unpin_map(obj);
+
+ err = i915_gem_object_set_to_gtt_domain(obj, false);
+ if (err)
+ goto err;
+
+ vma = i915_vma_instance(obj, vma->vm, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto err;
+ }
+
+ err = i915_vma_pin(vma, 0, 0, PIN_USER);
+ if (err)
+ goto err;
+
+ return vma;
+
+err:
+ i915_gem_object_put(obj);
+ return ERR_PTR(err);
+}
+
+static unsigned long real_page_count(struct drm_i915_gem_object *obj)
+{
+ return huge_gem_object_phys_size(obj) >> PAGE_SHIFT;
+}
+
+static unsigned long fake_page_count(struct drm_i915_gem_object *obj)
+{
+ return huge_gem_object_dma_size(obj) >> PAGE_SHIFT;
+}
+
+static int gpu_fill(struct drm_i915_gem_object *obj,
+ struct i915_gem_context *ctx,
+ struct intel_engine_cs *engine,
+ unsigned int dw)
+{
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+ struct i915_address_space *vm =
+ ctx->ppgtt ? &ctx->ppgtt->base : &i915->ggtt.base;
+ struct drm_i915_gem_request *rq;
+ struct i915_vma *vma;
+ struct i915_vma *batch;
+ unsigned int flags;
+ int err;
+
+ GEM_BUG_ON(obj->base.size > vm->total);
+
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma))
+ return PTR_ERR(vma);
+
+ err = i915_gem_object_set_to_gtt_domain(obj, false);
+ if (err)
+ return err;
+
+ err = i915_vma_pin(vma, 0, 0, PIN_HIGH | PIN_USER);
+ if (err)
+ return err;
+
+ /* Within the GTT the huge objects maps every page onto
+ * its 1024 real pages (using phys_pfn = dma_pfn % 1024).
+ * We set the nth dword within the page using the nth
+ * mapping via the GTT - this should exercise the GTT mapping
+ * whilst checking that each context provides a unique view
+ * into the object.
+ */
+ batch = gpu_fill_dw(vma,
+ (dw * real_page_count(obj)) << PAGE_SHIFT |
+ (dw * sizeof(u32)),
+ real_page_count(obj),
+ dw);
+ if (IS_ERR(batch)) {
+ err = PTR_ERR(batch);
+ goto err_vma;
+ }
+
+ rq = i915_gem_request_alloc(engine, ctx);
+ if (IS_ERR(rq)) {
+ err = PTR_ERR(rq);
+ goto err_batch;
+ }
+
+ err = engine->emit_flush(rq, EMIT_INVALIDATE);
+ if (err)
+ goto err_request;
+
+ err = i915_switch_context(rq);
+ if (err)
+ goto err_request;
+
+ flags = 0;
+ if (INTEL_GEN(vm->i915) <= 5)
+ flags |= I915_DISPATCH_SECURE;
+
+ err = engine->emit_bb_start(rq,
+ batch->node.start, batch->node.size,
+ flags);
+ if (err)
+ goto err_request;
+
+ i915_vma_move_to_active(batch, rq, 0);
+ i915_gem_object_set_active_reference(batch->obj);
+ i915_vma_unpin(batch);
+ i915_vma_close(batch);
+
+ i915_vma_move_to_active(vma, rq, 0);
+ i915_vma_unpin(vma);
+
+ reservation_object_lock(obj->resv, NULL);
+ reservation_object_add_excl_fence(obj->resv, &rq->fence);
+ reservation_object_unlock(obj->resv);
+
+ __i915_add_request(rq, true);
+
+ return 0;
+
+err_request:
+ __i915_add_request(rq, false);
+err_batch:
+ i915_vma_unpin(batch);
+err_vma:
+ i915_vma_unpin(vma);
+ return err;
+}
+
+static int cpu_fill(struct drm_i915_gem_object *obj, u32 value)
+{
+ const bool has_llc = HAS_LLC(to_i915(obj->base.dev));
+ unsigned int n, m, need_flush;
+ int err;
+
+ err = i915_gem_obj_prepare_shmem_write(obj, &need_flush);
+ if (err)
+ return err;
+
+ for (n = 0; n < real_page_count(obj); n++) {
+ u32 *map;
+
+ map = kmap_atomic(i915_gem_object_get_page(obj, n));
+ for (m = 0; m < DW_PER_PAGE; m++)
+ map[m] = value;
+ if (!has_llc)
+ drm_clflush_virt_range(map, PAGE_SIZE);
+ kunmap_atomic(map);
+ }
+
+ i915_gem_obj_finish_shmem_access(obj);
+ obj->base.read_domains = I915_GEM_DOMAIN_GTT | I915_GEM_DOMAIN_CPU;
+ obj->base.write_domain = 0;
+ return 0;
+}
+
+static int cpu_check(struct drm_i915_gem_object *obj, unsigned int max)
+{
+ unsigned int n, m, needs_flush;
+ int err;
+
+ err = i915_gem_obj_prepare_shmem_read(obj, &needs_flush);
+ if (err)
+ return err;
+
+ for (n = 0; n < real_page_count(obj); n++) {
+ u32 *map;
+
+ map = kmap_atomic(i915_gem_object_get_page(obj, n));
+ if (needs_flush & CLFLUSH_BEFORE)
+ drm_clflush_virt_range(map, PAGE_SIZE);
+
+ for (m = 0; m < max; m++) {
+ if (map[m] != m) {
+ pr_err("Invalid value at page %d, offset %d: found %x expected %x\n",
+ n, m, map[m], m);
+ err = -EINVAL;
+ goto out_unmap;
+ }
+ }
+
+ for (; m < DW_PER_PAGE; m++) {
+ if (map[m] != 0xdeadbeef) {
+ pr_err("Invalid value at page %d, offset %d: found %x expected %x\n",
+ n, m, map[m], 0xdeadbeef);
+ err = -EINVAL;
+ goto out_unmap;
+ }
+ }
+
+out_unmap:
+ kunmap_atomic(map);
+ if (err)
+ break;
+ }
+
+ i915_gem_obj_finish_shmem_access(obj);
+ return err;
+}
+
+static struct drm_i915_gem_object *
+create_test_object(struct i915_gem_context *ctx,
+ struct drm_file *file,
+ struct list_head *objects)
+{
+ struct drm_i915_gem_object *obj;
+ struct i915_address_space *vm =
+ ctx->ppgtt ? &ctx->ppgtt->base : &ctx->i915->ggtt.base;
+ u64 size;
+ u32 handle;
+ int err;
+
+ size = min(vm->total / 2, 1024ull * DW_PER_PAGE * PAGE_SIZE);
+ size = round_down(size, DW_PER_PAGE * PAGE_SIZE);
+
+ obj = huge_gem_object(ctx->i915, DW_PER_PAGE * PAGE_SIZE, size);
+ if (IS_ERR(obj))
+ return obj;
+
+ /* tie the handle to the drm_file for easy reaping */
+ err = drm_gem_handle_create(file, &obj->base, &handle);
+ i915_gem_object_put(obj);
+ if (err)
+ return ERR_PTR(err);
+
+ err = cpu_fill(obj, 0xdeadbeef);
+ if (err) {
+ pr_err("Failed to fill object with cpu, err=%d\n",
+ err);
+ return ERR_PTR(err);
+ }
+
+ list_add_tail(&obj->st_link, objects);
+ return obj;
+}
+
+static unsigned long max_dwords(struct drm_i915_gem_object *obj)
+{
+ unsigned long npages = fake_page_count(obj);
+
+ GEM_BUG_ON(!IS_ALIGNED(npages, DW_PER_PAGE));
+ return npages / DW_PER_PAGE;
+}
+
+static int igt_ctx_exec(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj;
+ struct drm_file *file;
+ IGT_TIMEOUT(end_time);
+ LIST_HEAD(objects);
+ unsigned long ncontexts, ndwords, dw;
+ bool first_shared_gtt = true;
+ int err;
+
+ /* Create a few different contexts (with different mm) and write
+ * through each ctx/mm using the GPU making sure those writes end
+ * up in the expected pages of our obj.
+ */
+
+ file = mock_file(i915);
+ if (IS_ERR(file))
+ return PTR_ERR(file);
+
+ mutex_lock(&i915->drm.struct_mutex);
+
+ ncontexts = 0;
+ ndwords = 0;
+ dw = 0;
+ while (!time_after(jiffies, end_time)) {
+ struct intel_engine_cs *engine;
+ struct i915_gem_context *ctx;
+ unsigned int id;
+
+ if (first_shared_gtt) {
+ ctx = __create_hw_context(i915, file->driver_priv);
+ first_shared_gtt = false;
+ } else {
+ ctx = i915_gem_create_context(i915, file->driver_priv);
+ }
+ if (IS_ERR(ctx)) {
+ err = PTR_ERR(ctx);
+ goto out_unlock;
+ }
+
+ for_each_engine(engine, i915, id) {
+ if (dw == 0) {
+ obj = create_test_object(ctx, file, &objects);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto out_unlock;
+ }
+ }
+
+ err = gpu_fill(obj, ctx, engine, dw);
+ if (err) {
+ pr_err("Failed to fill dword %lu [%lu/%lu] with gpu (%s) in ctx %u [full-ppgtt? %s], err=%d\n",
+ ndwords, dw, max_dwords(obj),
+ engine->name, ctx->hw_id,
+ yesno(!!ctx->ppgtt), err);
+ goto out_unlock;
+ }
+
+ if (++dw == max_dwords(obj))
+ dw = 0;
+ ndwords++;
+ }
+ ncontexts++;
+ }
+ pr_info("Submitted %lu contexts (across %u engines), filling %lu dwords\n",
+ ncontexts, INTEL_INFO(i915)->num_rings, ndwords);
+
+ dw = 0;
+ list_for_each_entry(obj, &objects, st_link) {
+ unsigned int rem =
+ min_t(unsigned int, ndwords - dw, max_dwords(obj));
+
+ err = cpu_check(obj, rem);
+ if (err)
+ break;
+
+ dw += rem;
+ }
+
+out_unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ mock_file_free(i915, file);
+ return err;
+}
+
+static int fake_aliasing_ppgtt_enable(struct drm_i915_private *i915)
+{
+ struct drm_i915_gem_object *obj;
+ int err;
+
+ err = i915_gem_init_aliasing_ppgtt(i915);
+ if (err)
+ return err;
+
+ list_for_each_entry(obj, &i915->mm.bound_list, global_link) {
+ struct i915_vma *vma;
+
+ vma = i915_vma_instance(obj, &i915->ggtt.base, NULL);
+ if (IS_ERR(vma))
+ continue;
+
+ vma->flags &= ~I915_VMA_LOCAL_BIND;
+ }
+
+ return 0;
+}
+
+static void fake_aliasing_ppgtt_disable(struct drm_i915_private *i915)
+{
+ i915_gem_fini_aliasing_ppgtt(i915);
+}
+
+int i915_gem_context_live_selftests(struct drm_i915_private *dev_priv)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_ctx_exec),
+ };
+ bool fake_alias = false;
+ int err;
+
+ /* Install a fake aliasing gtt for exercise */
+ if (USES_PPGTT(dev_priv) && !dev_priv->mm.aliasing_ppgtt) {
+ mutex_lock(&dev_priv->drm.struct_mutex);
+ err = fake_aliasing_ppgtt_enable(dev_priv);
+ mutex_unlock(&dev_priv->drm.struct_mutex);
+ if (err)
+ return err;
+
+ GEM_BUG_ON(!dev_priv->mm.aliasing_ppgtt);
+ fake_alias = true;
+ }
+
+ err = i915_subtests(tests, dev_priv);
+
+ if (fake_alias) {
+ mutex_lock(&dev_priv->drm.struct_mutex);
+ fake_aliasing_ppgtt_disable(dev_priv);
+ mutex_unlock(&dev_priv->drm.struct_mutex);
+ }
+
+ return err;
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "../i915_selftest.h"
+
+#include "mock_gem_device.h"
+#include "mock_dmabuf.h"
+
+static int igt_dmabuf_export(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj;
+ struct dma_buf *dmabuf;
+
+ obj = i915_gem_object_create(i915, PAGE_SIZE);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ dmabuf = i915_gem_prime_export(&i915->drm, &obj->base, 0);
+ i915_gem_object_put(obj);
+ if (IS_ERR(dmabuf)) {
+ pr_err("i915_gem_prime_export failed with err=%d\n",
+ (int)PTR_ERR(dmabuf));
+ return PTR_ERR(dmabuf);
+ }
+
+ dma_buf_put(dmabuf);
+ return 0;
+}
+
+static int igt_dmabuf_import_self(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj;
+ struct drm_gem_object *import;
+ struct dma_buf *dmabuf;
+ int err;
+
+ obj = i915_gem_object_create(i915, PAGE_SIZE);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ dmabuf = i915_gem_prime_export(&i915->drm, &obj->base, 0);
+ if (IS_ERR(dmabuf)) {
+ pr_err("i915_gem_prime_export failed with err=%d\n",
+ (int)PTR_ERR(dmabuf));
+ err = PTR_ERR(dmabuf);
+ goto out;
+ }
+
+ import = i915_gem_prime_import(&i915->drm, dmabuf);
+ if (IS_ERR(import)) {
+ pr_err("i915_gem_prime_import failed with err=%d\n",
+ (int)PTR_ERR(import));
+ err = PTR_ERR(import);
+ goto out_dmabuf;
+ }
+
+ if (import != &obj->base) {
+ pr_err("i915_gem_prime_import created a new object!\n");
+ err = -EINVAL;
+ goto out_import;
+ }
+
+ err = 0;
+out_import:
+ i915_gem_object_put(to_intel_bo(import));
+out_dmabuf:
+ dma_buf_put(dmabuf);
+out:
+ i915_gem_object_put(obj);
+ return err;
+}
+
+static int igt_dmabuf_import(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj;
+ struct dma_buf *dmabuf;
+ void *obj_map, *dma_map;
+ u32 pattern[] = { 0, 0xaa, 0xcc, 0x55, 0xff };
+ int err, i;
+
+ dmabuf = mock_dmabuf(1);
+ if (IS_ERR(dmabuf))
+ return PTR_ERR(dmabuf);
+
+ obj = to_intel_bo(i915_gem_prime_import(&i915->drm, dmabuf));
+ if (IS_ERR(obj)) {
+ pr_err("i915_gem_prime_import failed with err=%d\n",
+ (int)PTR_ERR(obj));
+ err = PTR_ERR(obj);
+ goto out_dmabuf;
+ }
+
+ if (obj->base.dev != &i915->drm) {
+ pr_err("i915_gem_prime_import created a non-i915 object!\n");
+ err = -EINVAL;
+ goto out_obj;
+ }
+
+ if (obj->base.size != PAGE_SIZE) {
+ pr_err("i915_gem_prime_import is wrong size found %lld, expected %ld\n",
+ (long long)obj->base.size, PAGE_SIZE);
+ err = -EINVAL;
+ goto out_obj;
+ }
+
+ dma_map = dma_buf_vmap(dmabuf);
+ if (!dma_map) {
+ pr_err("dma_buf_vmap failed\n");
+ err = -ENOMEM;
+ goto out_obj;
+ }
+
+ if (0) { /* Can not yet map dmabuf */
+ obj_map = i915_gem_object_pin_map(obj, I915_MAP_WB);
+ if (IS_ERR(obj_map)) {
+ err = PTR_ERR(obj_map);
+ pr_err("i915_gem_object_pin_map failed with err=%d\n", err);
+ goto out_dma_map;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(pattern); i++) {
+ memset(dma_map, pattern[i], PAGE_SIZE);
+ if (memchr_inv(obj_map, pattern[i], PAGE_SIZE)) {
+ err = -EINVAL;
+ pr_err("imported vmap not all set to %x!\n", pattern[i]);
+ i915_gem_object_unpin_map(obj);
+ goto out_dma_map;
+ }
+ }
+
+ for (i = 0; i < ARRAY_SIZE(pattern); i++) {
+ memset(obj_map, pattern[i], PAGE_SIZE);
+ if (memchr_inv(dma_map, pattern[i], PAGE_SIZE)) {
+ err = -EINVAL;
+ pr_err("exported vmap not all set to %x!\n", pattern[i]);
+ i915_gem_object_unpin_map(obj);
+ goto out_dma_map;
+ }
+ }
+
+ i915_gem_object_unpin_map(obj);
+ }
+
+ err = 0;
+out_dma_map:
+ dma_buf_vunmap(dmabuf, dma_map);
+out_obj:
+ i915_gem_object_put(obj);
+out_dmabuf:
+ dma_buf_put(dmabuf);
+ return err;
+}
+
+static int igt_dmabuf_import_ownership(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj;
+ struct dma_buf *dmabuf;
+ void *ptr;
+ int err;
+
+ dmabuf = mock_dmabuf(1);
+ if (IS_ERR(dmabuf))
+ return PTR_ERR(dmabuf);
+
+ ptr = dma_buf_vmap(dmabuf);
+ if (!ptr) {
+ pr_err("dma_buf_vmap failed\n");
+ err = -ENOMEM;
+ goto err_dmabuf;
+ }
+
+ memset(ptr, 0xc5, PAGE_SIZE);
+ dma_buf_vunmap(dmabuf, ptr);
+
+ obj = to_intel_bo(i915_gem_prime_import(&i915->drm, dmabuf));
+ if (IS_ERR(obj)) {
+ pr_err("i915_gem_prime_import failed with err=%d\n",
+ (int)PTR_ERR(obj));
+ err = PTR_ERR(obj);
+ goto err_dmabuf;
+ }
+
+ dma_buf_put(dmabuf);
+
+ err = i915_gem_object_pin_pages(obj);
+ if (err) {
+ pr_err("i915_gem_object_pin_pages failed with err=%d\n", err);
+ goto out_obj;
+ }
+
+ err = 0;
+ i915_gem_object_unpin_pages(obj);
+out_obj:
+ i915_gem_object_put(obj);
+ return err;
+
+err_dmabuf:
+ dma_buf_put(dmabuf);
+ return err;
+}
+
+static int igt_dmabuf_export_vmap(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj;
+ struct dma_buf *dmabuf;
+ void *ptr;
+ int err;
+
+ obj = i915_gem_object_create(i915, PAGE_SIZE);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ dmabuf = i915_gem_prime_export(&i915->drm, &obj->base, 0);
+ if (IS_ERR(dmabuf)) {
+ pr_err("i915_gem_prime_export failed with err=%d\n",
+ (int)PTR_ERR(dmabuf));
+ err = PTR_ERR(dmabuf);
+ goto err_obj;
+ }
+ i915_gem_object_put(obj);
+
+ ptr = dma_buf_vmap(dmabuf);
+ if (IS_ERR(ptr)) {
+ err = PTR_ERR(ptr);
+ pr_err("dma_buf_vmap failed with err=%d\n", err);
+ goto out;
+ }
+
+ if (memchr_inv(ptr, 0, dmabuf->size)) {
+ pr_err("Exported object not initialiased to zero!\n");
+ err = -EINVAL;
+ goto out;
+ }
+
+ memset(ptr, 0xc5, dmabuf->size);
+
+ err = 0;
+ dma_buf_vunmap(dmabuf, ptr);
+out:
+ dma_buf_put(dmabuf);
+ return err;
+
+err_obj:
+ i915_gem_object_put(obj);
+ return err;
+}
+
+int i915_gem_dmabuf_mock_selftests(void)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_dmabuf_export),
+ SUBTEST(igt_dmabuf_import_self),
+ SUBTEST(igt_dmabuf_import),
+ SUBTEST(igt_dmabuf_import_ownership),
+ SUBTEST(igt_dmabuf_export_vmap),
+ };
+ struct drm_i915_private *i915;
+ int err;
+
+ i915 = mock_gem_device();
+ if (!i915)
+ return -ENOMEM;
+
+ err = i915_subtests(tests, i915);
+
+ drm_dev_unref(&i915->drm);
+ return err;
+}
+
+int i915_gem_dmabuf_live_selftests(struct drm_i915_private *i915)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_dmabuf_export),
+ };
+
+ return i915_subtests(tests, i915);
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "../i915_selftest.h"
+
+#include "mock_gem_device.h"
+
+static int populate_ggtt(struct drm_i915_private *i915)
+{
+ struct drm_i915_gem_object *obj;
+ u64 size;
+
+ for (size = 0;
+ size + I915_GTT_PAGE_SIZE <= i915->ggtt.base.total;
+ size += I915_GTT_PAGE_SIZE) {
+ struct i915_vma *vma;
+
+ obj = i915_gem_object_create_internal(i915, I915_GTT_PAGE_SIZE);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0, 0);
+ if (IS_ERR(vma))
+ return PTR_ERR(vma);
+ }
+
+ if (!list_empty(&i915->mm.unbound_list)) {
+ size = 0;
+ list_for_each_entry(obj, &i915->mm.unbound_list, global_link)
+ size++;
+
+ pr_err("Found %lld objects unbound!\n", size);
+ return -EINVAL;
+ }
+
+ if (list_empty(&i915->ggtt.base.inactive_list)) {
+ pr_err("No objects on the GGTT inactive list!\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void unpin_ggtt(struct drm_i915_private *i915)
+{
+ struct i915_vma *vma;
+
+ list_for_each_entry(vma, &i915->ggtt.base.inactive_list, vm_link)
+ i915_vma_unpin(vma);
+}
+
+static void cleanup_objects(struct drm_i915_private *i915)
+{
+ struct drm_i915_gem_object *obj, *on;
+
+ list_for_each_entry_safe(obj, on, &i915->mm.unbound_list, global_link)
+ i915_gem_object_put(obj);
+
+ list_for_each_entry_safe(obj, on, &i915->mm.bound_list, global_link)
+ i915_gem_object_put(obj);
+
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ i915_gem_drain_freed_objects(i915);
+
+ mutex_lock(&i915->drm.struct_mutex);
+}
+
+static int igt_evict_something(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct i915_ggtt *ggtt = &i915->ggtt;
+ int err;
+
+ /* Fill the GGTT with pinned objects and try to evict one. */
+
+ err = populate_ggtt(i915);
+ if (err)
+ goto cleanup;
+
+ /* Everything is pinned, nothing should happen */
+ err = i915_gem_evict_something(&ggtt->base,
+ I915_GTT_PAGE_SIZE, 0, 0,
+ 0, U64_MAX,
+ 0);
+ if (err != -ENOSPC) {
+ pr_err("i915_gem_evict_something failed on a full GGTT with err=%d\n",
+ err);
+ goto cleanup;
+ }
+
+ unpin_ggtt(i915);
+
+ /* Everything is unpinned, we should be able to evict something */
+ err = i915_gem_evict_something(&ggtt->base,
+ I915_GTT_PAGE_SIZE, 0, 0,
+ 0, U64_MAX,
+ 0);
+ if (err) {
+ pr_err("i915_gem_evict_something failed on a full GGTT with err=%d\n",
+ err);
+ goto cleanup;
+ }
+
+cleanup:
+ cleanup_objects(i915);
+ return err;
+}
+
+static int igt_overcommit(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj;
+ struct i915_vma *vma;
+ int err;
+
+ /* Fill the GGTT with pinned objects and then try to pin one more.
+ * We expect it to fail.
+ */
+
+ err = populate_ggtt(i915);
+ if (err)
+ goto cleanup;
+
+ obj = i915_gem_object_create_internal(i915, I915_GTT_PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto cleanup;
+ }
+
+ list_move(&obj->global_link, &i915->mm.unbound_list);
+
+ vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0, 0);
+ if (!IS_ERR(vma) || PTR_ERR(vma) != -ENOSPC) {
+ pr_err("Failed to evict+insert, i915_gem_object_ggtt_pin returned err=%d\n", (int)PTR_ERR(vma));
+ err = -EINVAL;
+ goto cleanup;
+ }
+
+cleanup:
+ cleanup_objects(i915);
+ return err;
+}
+
+static int igt_evict_for_vma(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct i915_ggtt *ggtt = &i915->ggtt;
+ struct drm_mm_node target = {
+ .start = 0,
+ .size = 4096,
+ };
+ int err;
+
+ /* Fill the GGTT with pinned objects and try to evict a range. */
+
+ err = populate_ggtt(i915);
+ if (err)
+ goto cleanup;
+
+ /* Everything is pinned, nothing should happen */
+ err = i915_gem_evict_for_node(&ggtt->base, &target, 0);
+ if (err != -ENOSPC) {
+ pr_err("i915_gem_evict_for_node on a full GGTT returned err=%d\n",
+ err);
+ goto cleanup;
+ }
+
+ unpin_ggtt(i915);
+
+ /* Everything is unpinned, we should be able to evict the node */
+ err = i915_gem_evict_for_node(&ggtt->base, &target, 0);
+ if (err) {
+ pr_err("i915_gem_evict_for_node returned err=%d\n",
+ err);
+ goto cleanup;
+ }
+
+cleanup:
+ cleanup_objects(i915);
+ return err;
+}
+
+static void mock_color_adjust(const struct drm_mm_node *node,
+ unsigned long color,
+ u64 *start,
+ u64 *end)
+{
+}
+
+static int igt_evict_for_cache_color(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct i915_ggtt *ggtt = &i915->ggtt;
+ const unsigned long flags = PIN_OFFSET_FIXED;
+ struct drm_mm_node target = {
+ .start = I915_GTT_PAGE_SIZE * 2,
+ .size = I915_GTT_PAGE_SIZE,
+ .color = I915_CACHE_LLC,
+ };
+ struct drm_i915_gem_object *obj;
+ struct i915_vma *vma;
+ int err;
+
+ /* Currently the use of color_adjust is limited to cache domains within
+ * the ggtt, and so the presence of mm.color_adjust is assumed to be
+ * i915_gtt_color_adjust throughout our driver, so using a mock color
+ * adjust will work just fine for our purposes.
+ */
+ ggtt->base.mm.color_adjust = mock_color_adjust;
+
+ obj = i915_gem_object_create_internal(i915, I915_GTT_PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto cleanup;
+ }
+ i915_gem_object_set_cache_level(obj, I915_CACHE_LLC);
+
+ vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0,
+ I915_GTT_PAGE_SIZE | flags);
+ if (IS_ERR(vma)) {
+ pr_err("[0]i915_gem_object_ggtt_pin failed\n");
+ err = PTR_ERR(vma);
+ goto cleanup;
+ }
+
+ obj = i915_gem_object_create_internal(i915, I915_GTT_PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto cleanup;
+ }
+ i915_gem_object_set_cache_level(obj, I915_CACHE_LLC);
+
+ /* Neighbouring; same colour - should fit */
+ vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0,
+ (I915_GTT_PAGE_SIZE * 2) | flags);
+ if (IS_ERR(vma)) {
+ pr_err("[1]i915_gem_object_ggtt_pin failed\n");
+ err = PTR_ERR(vma);
+ goto cleanup;
+ }
+
+ i915_vma_unpin(vma);
+
+ /* Remove just the second vma */
+ err = i915_gem_evict_for_node(&ggtt->base, &target, 0);
+ if (err) {
+ pr_err("[0]i915_gem_evict_for_node returned err=%d\n", err);
+ goto cleanup;
+ }
+
+ /* Attempt to remove the first *pinned* vma, by removing the (empty)
+ * neighbour -- this should fail.
+ */
+ target.color = I915_CACHE_L3_LLC;
+
+ err = i915_gem_evict_for_node(&ggtt->base, &target, 0);
+ if (!err) {
+ pr_err("[1]i915_gem_evict_for_node returned err=%d\n", err);
+ err = -EINVAL;
+ goto cleanup;
+ }
+
+ err = 0;
+
+cleanup:
+ unpin_ggtt(i915);
+ cleanup_objects(i915);
+ ggtt->base.mm.color_adjust = NULL;
+ return err;
+}
+
+static int igt_evict_vm(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct i915_ggtt *ggtt = &i915->ggtt;
+ int err;
+
+ /* Fill the GGTT with pinned objects and try to evict everything. */
+
+ err = populate_ggtt(i915);
+ if (err)
+ goto cleanup;
+
+ /* Everything is pinned, nothing should happen */
+ err = i915_gem_evict_vm(&ggtt->base, false);
+ if (err) {
+ pr_err("i915_gem_evict_vm on a full GGTT returned err=%d]\n",
+ err);
+ goto cleanup;
+ }
+
+ unpin_ggtt(i915);
+
+ err = i915_gem_evict_vm(&ggtt->base, false);
+ if (err) {
+ pr_err("i915_gem_evict_vm on a full GGTT returned err=%d]\n",
+ err);
+ goto cleanup;
+ }
+
+cleanup:
+ cleanup_objects(i915);
+ return err;
+}
+
+int i915_gem_evict_mock_selftests(void)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_evict_something),
+ SUBTEST(igt_evict_for_vma),
+ SUBTEST(igt_evict_for_cache_color),
+ SUBTEST(igt_evict_vm),
+ SUBTEST(igt_overcommit),
+ };
+ struct drm_i915_private *i915;
+ int err;
+
+ i915 = mock_gem_device();
+ if (!i915)
+ return -ENOMEM;
+
+ mutex_lock(&i915->drm.struct_mutex);
+ err = i915_subtests(tests, i915);
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ drm_dev_unref(&i915->drm);
+ return err;
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include <linux/list_sort.h>
+#include <linux/prime_numbers.h>
+
+#include "../i915_selftest.h"
+#include "i915_random.h"
+
+#include "mock_context.h"
+#include "mock_drm.h"
+#include "mock_gem_device.h"
+
+static void fake_free_pages(struct drm_i915_gem_object *obj,
+ struct sg_table *pages)
+{
+ sg_free_table(pages);
+ kfree(pages);
+}
+
+static struct sg_table *
+fake_get_pages(struct drm_i915_gem_object *obj)
+{
+#define GFP (GFP_KERNEL | __GFP_NOWARN | __GFP_NORETRY)
+#define PFN_BIAS 0x1000
+ struct sg_table *pages;
+ struct scatterlist *sg;
+ typeof(obj->base.size) rem;
+
+ pages = kmalloc(sizeof(*pages), GFP);
+ if (!pages)
+ return ERR_PTR(-ENOMEM);
+
+ rem = round_up(obj->base.size, BIT(31)) >> 31;
+ if (sg_alloc_table(pages, rem, GFP)) {
+ kfree(pages);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ rem = obj->base.size;
+ for (sg = pages->sgl; sg; sg = sg_next(sg)) {
+ unsigned long len = min_t(typeof(rem), rem, BIT(31));
+
+ GEM_BUG_ON(!len);
+ sg_set_page(sg, pfn_to_page(PFN_BIAS), len, 0);
+ sg_dma_address(sg) = page_to_phys(sg_page(sg));
+ sg_dma_len(sg) = len;
+
+ rem -= len;
+ }
+ GEM_BUG_ON(rem);
+
+ obj->mm.madv = I915_MADV_DONTNEED;
+ return pages;
+#undef GFP
+}
+
+static void fake_put_pages(struct drm_i915_gem_object *obj,
+ struct sg_table *pages)
+{
+ fake_free_pages(obj, pages);
+ obj->mm.dirty = false;
+ obj->mm.madv = I915_MADV_WILLNEED;
+}
+
+static const struct drm_i915_gem_object_ops fake_ops = {
+ .flags = I915_GEM_OBJECT_IS_SHRINKABLE,
+ .get_pages = fake_get_pages,
+ .put_pages = fake_put_pages,
+};
+
+static struct drm_i915_gem_object *
+fake_dma_object(struct drm_i915_private *i915, u64 size)
+{
+ struct drm_i915_gem_object *obj;
+
+ GEM_BUG_ON(!size);
+ GEM_BUG_ON(!IS_ALIGNED(size, I915_GTT_PAGE_SIZE));
+
+ if (overflows_type(size, obj->base.size))
+ return ERR_PTR(-E2BIG);
+
+ obj = i915_gem_object_alloc(i915);
+ if (!obj)
+ goto err;
+
+ drm_gem_private_object_init(&i915->drm, &obj->base, size);
+ i915_gem_object_init(obj, &fake_ops);
+
+ obj->base.write_domain = I915_GEM_DOMAIN_CPU;
+ obj->base.read_domains = I915_GEM_DOMAIN_CPU;
+ obj->cache_level = I915_CACHE_NONE;
+
+ /* Preallocate the "backing storage" */
+ if (i915_gem_object_pin_pages(obj))
+ goto err_obj;
+
+ i915_gem_object_unpin_pages(obj);
+ return obj;
+
+err_obj:
+ i915_gem_object_put(obj);
+err:
+ return ERR_PTR(-ENOMEM);
+}
+
+static int igt_ppgtt_alloc(void *arg)
+{
+ struct drm_i915_private *dev_priv = arg;
+ struct i915_hw_ppgtt *ppgtt;
+ u64 size, last;
+ int err;
+
+ /* Allocate a ppggt and try to fill the entire range */
+
+ if (!USES_PPGTT(dev_priv))
+ return 0;
+
+ ppgtt = kzalloc(sizeof(*ppgtt), GFP_KERNEL);
+ if (!ppgtt)
+ return -ENOMEM;
+
+ mutex_lock(&dev_priv->drm.struct_mutex);
+ err = __hw_ppgtt_init(ppgtt, dev_priv);
+ if (err)
+ goto err_ppgtt;
+
+ if (!ppgtt->base.allocate_va_range)
+ goto err_ppgtt_cleanup;
+
+ /* Check we can allocate the entire range */
+ for (size = 4096;
+ size <= ppgtt->base.total;
+ size <<= 2) {
+ err = ppgtt->base.allocate_va_range(&ppgtt->base, 0, size);
+ if (err) {
+ if (err == -ENOMEM) {
+ pr_info("[1] Ran out of memory for va_range [0 + %llx] [bit %d]\n",
+ size, ilog2(size));
+ err = 0; /* virtual space too large! */
+ }
+ goto err_ppgtt_cleanup;
+ }
+
+ ppgtt->base.clear_range(&ppgtt->base, 0, size);
+ }
+
+ /* Check we can incrementally allocate the entire range */
+ for (last = 0, size = 4096;
+ size <= ppgtt->base.total;
+ last = size, size <<= 2) {
+ err = ppgtt->base.allocate_va_range(&ppgtt->base,
+ last, size - last);
+ if (err) {
+ if (err == -ENOMEM) {
+ pr_info("[2] Ran out of memory for va_range [%llx + %llx] [bit %d]\n",
+ last, size - last, ilog2(size));
+ err = 0; /* virtual space too large! */
+ }
+ goto err_ppgtt_cleanup;
+ }
+ }
+
+err_ppgtt_cleanup:
+ ppgtt->base.cleanup(&ppgtt->base);
+err_ppgtt:
+ mutex_unlock(&dev_priv->drm.struct_mutex);
+ kfree(ppgtt);
+ return err;
+}
+
+static int lowlevel_hole(struct drm_i915_private *i915,
+ struct i915_address_space *vm,
+ u64 hole_start, u64 hole_end,
+ unsigned long end_time)
+{
+ I915_RND_STATE(seed_prng);
+ unsigned int size;
+
+ /* Keep creating larger objects until one cannot fit into the hole */
+ for (size = 12; (hole_end - hole_start) >> size; size++) {
+ I915_RND_SUBSTATE(prng, seed_prng);
+ struct drm_i915_gem_object *obj;
+ unsigned int *order, count, n;
+ u64 hole_size;
+
+ hole_size = (hole_end - hole_start) >> size;
+ if (hole_size > KMALLOC_MAX_SIZE / sizeof(u32))
+ hole_size = KMALLOC_MAX_SIZE / sizeof(u32);
+ count = hole_size;
+ do {
+ count >>= 1;
+ order = i915_random_order(count, &prng);
+ } while (!order && count);
+ if (!order)
+ break;
+
+ GEM_BUG_ON(count * BIT_ULL(size) > vm->total);
+ GEM_BUG_ON(hole_start + count * BIT_ULL(size) > hole_end);
+
+ /* Ignore allocation failures (i.e. don't report them as
+ * a test failure) as we are purposefully allocating very
+ * large objects without checking that we have sufficient
+ * memory. We expect to hit -ENOMEM.
+ */
+
+ obj = fake_dma_object(i915, BIT_ULL(size));
+ if (IS_ERR(obj)) {
+ kfree(order);
+ break;
+ }
+
+ GEM_BUG_ON(obj->base.size != BIT_ULL(size));
+
+ if (i915_gem_object_pin_pages(obj)) {
+ i915_gem_object_put(obj);
+ kfree(order);
+ break;
+ }
+
+ for (n = 0; n < count; n++) {
+ u64 addr = hole_start + order[n] * BIT_ULL(size);
+
+ GEM_BUG_ON(addr + BIT_ULL(size) > vm->total);
+
+ if (igt_timeout(end_time,
+ "%s timed out before %d/%d\n",
+ __func__, n, count)) {
+ hole_end = hole_start; /* quit */
+ break;
+ }
+
+ if (vm->allocate_va_range &&
+ vm->allocate_va_range(vm, addr, BIT_ULL(size)))
+ break;
+
+ vm->insert_entries(vm, obj->mm.pages, addr,
+ I915_CACHE_NONE, 0);
+ }
+ count = n;
+
+ i915_random_reorder(order, count, &prng);
+ for (n = 0; n < count; n++) {
+ u64 addr = hole_start + order[n] * BIT_ULL(size);
+
+ GEM_BUG_ON(addr + BIT_ULL(size) > vm->total);
+ vm->clear_range(vm, addr, BIT_ULL(size));
+ }
+
+ i915_gem_object_unpin_pages(obj);
+ i915_gem_object_put(obj);
+
+ kfree(order);
+ }
+
+ return 0;
+}
+
+static void close_object_list(struct list_head *objects,
+ struct i915_address_space *vm)
+{
+ struct drm_i915_gem_object *obj, *on;
+ int ignored;
+
+ list_for_each_entry_safe(obj, on, objects, st_link) {
+ struct i915_vma *vma;
+
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (!IS_ERR(vma))
+ ignored = i915_vma_unbind(vma);
+ /* Only ppgtt vma may be closed before the object is freed */
+ if (!IS_ERR(vma) && !i915_vma_is_ggtt(vma))
+ i915_vma_close(vma);
+
+ list_del(&obj->st_link);
+ i915_gem_object_put(obj);
+ }
+}
+
+static int fill_hole(struct drm_i915_private *i915,
+ struct i915_address_space *vm,
+ u64 hole_start, u64 hole_end,
+ unsigned long end_time)
+{
+ const u64 hole_size = hole_end - hole_start;
+ struct drm_i915_gem_object *obj;
+ const unsigned long max_pages =
+ min_t(u64, ULONG_MAX - 1, hole_size/2 >> PAGE_SHIFT);
+ const unsigned long max_step = max(int_sqrt(max_pages), 2UL);
+ unsigned long npages, prime, flags;
+ struct i915_vma *vma;
+ LIST_HEAD(objects);
+ int err;
+
+ /* Try binding many VMA working inwards from either edge */
+
+ flags = PIN_OFFSET_FIXED | PIN_USER;
+ if (i915_is_ggtt(vm))
+ flags |= PIN_GLOBAL;
+
+ for_each_prime_number_from(prime, 2, max_step) {
+ for (npages = 1; npages <= max_pages; npages *= prime) {
+ const u64 full_size = npages << PAGE_SHIFT;
+ const struct {
+ const char *name;
+ u64 offset;
+ int step;
+ } phases[] = {
+ { "top-down", hole_end, -1, },
+ { "bottom-up", hole_start, 1, },
+ { }
+ }, *p;
+
+ obj = fake_dma_object(i915, full_size);
+ if (IS_ERR(obj))
+ break;
+
+ list_add(&obj->st_link, &objects);
+
+ /* Align differing sized objects against the edges, and
+ * check we don't walk off into the void when binding
+ * them into the GTT.
+ */
+ for (p = phases; p->name; p++) {
+ u64 offset;
+
+ offset = p->offset;
+ list_for_each_entry(obj, &objects, st_link) {
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma))
+ continue;
+
+ if (p->step < 0) {
+ if (offset < hole_start + obj->base.size)
+ break;
+ offset -= obj->base.size;
+ }
+
+ err = i915_vma_pin(vma, 0, 0, offset | flags);
+ if (err) {
+ pr_err("%s(%s) pin (forward) failed with err=%d on size=%lu pages (prime=%lu), offset=%llx\n",
+ __func__, p->name, err, npages, prime, offset);
+ goto err;
+ }
+
+ if (!drm_mm_node_allocated(&vma->node) ||
+ i915_vma_misplaced(vma, 0, 0, offset | flags)) {
+ pr_err("%s(%s) (forward) insert failed: vma.node=%llx + %llx [allocated? %d], expected offset %llx\n",
+ __func__, p->name, vma->node.start, vma->node.size, drm_mm_node_allocated(&vma->node),
+ offset);
+ err = -EINVAL;
+ goto err;
+ }
+
+ i915_vma_unpin(vma);
+
+ if (p->step > 0) {
+ if (offset + obj->base.size > hole_end)
+ break;
+ offset += obj->base.size;
+ }
+ }
+
+ offset = p->offset;
+ list_for_each_entry(obj, &objects, st_link) {
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma))
+ continue;
+
+ if (p->step < 0) {
+ if (offset < hole_start + obj->base.size)
+ break;
+ offset -= obj->base.size;
+ }
+
+ if (!drm_mm_node_allocated(&vma->node) ||
+ i915_vma_misplaced(vma, 0, 0, offset | flags)) {
+ pr_err("%s(%s) (forward) moved vma.node=%llx + %llx, expected offset %llx\n",
+ __func__, p->name, vma->node.start, vma->node.size,
+ offset);
+ err = -EINVAL;
+ goto err;
+ }
+
+ err = i915_vma_unbind(vma);
+ if (err) {
+ pr_err("%s(%s) (forward) unbind of vma.node=%llx + %llx failed with err=%d\n",
+ __func__, p->name, vma->node.start, vma->node.size,
+ err);
+ goto err;
+ }
+
+ if (p->step > 0) {
+ if (offset + obj->base.size > hole_end)
+ break;
+ offset += obj->base.size;
+ }
+ }
+
+ offset = p->offset;
+ list_for_each_entry_reverse(obj, &objects, st_link) {
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma))
+ continue;
+
+ if (p->step < 0) {
+ if (offset < hole_start + obj->base.size)
+ break;
+ offset -= obj->base.size;
+ }
+
+ err = i915_vma_pin(vma, 0, 0, offset | flags);
+ if (err) {
+ pr_err("%s(%s) pin (backward) failed with err=%d on size=%lu pages (prime=%lu), offset=%llx\n",
+ __func__, p->name, err, npages, prime, offset);
+ goto err;
+ }
+
+ if (!drm_mm_node_allocated(&vma->node) ||
+ i915_vma_misplaced(vma, 0, 0, offset | flags)) {
+ pr_err("%s(%s) (backward) insert failed: vma.node=%llx + %llx [allocated? %d], expected offset %llx\n",
+ __func__, p->name, vma->node.start, vma->node.size, drm_mm_node_allocated(&vma->node),
+ offset);
+ err = -EINVAL;
+ goto err;
+ }
+
+ i915_vma_unpin(vma);
+
+ if (p->step > 0) {
+ if (offset + obj->base.size > hole_end)
+ break;
+ offset += obj->base.size;
+ }
+ }
+
+ offset = p->offset;
+ list_for_each_entry_reverse(obj, &objects, st_link) {
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma))
+ continue;
+
+ if (p->step < 0) {
+ if (offset < hole_start + obj->base.size)
+ break;
+ offset -= obj->base.size;
+ }
+
+ if (!drm_mm_node_allocated(&vma->node) ||
+ i915_vma_misplaced(vma, 0, 0, offset | flags)) {
+ pr_err("%s(%s) (backward) moved vma.node=%llx + %llx [allocated? %d], expected offset %llx\n",
+ __func__, p->name, vma->node.start, vma->node.size, drm_mm_node_allocated(&vma->node),
+ offset);
+ err = -EINVAL;
+ goto err;
+ }
+
+ err = i915_vma_unbind(vma);
+ if (err) {
+ pr_err("%s(%s) (backward) unbind of vma.node=%llx + %llx failed with err=%d\n",
+ __func__, p->name, vma->node.start, vma->node.size,
+ err);
+ goto err;
+ }
+
+ if (p->step > 0) {
+ if (offset + obj->base.size > hole_end)
+ break;
+ offset += obj->base.size;
+ }
+ }
+ }
+
+ if (igt_timeout(end_time, "%s timed out (npages=%lu, prime=%lu)\n",
+ __func__, npages, prime)) {
+ err = -EINTR;
+ goto err;
+ }
+ }
+
+ close_object_list(&objects, vm);
+ }
+
+ return 0;
+
+err:
+ close_object_list(&objects, vm);
+ return err;
+}
+
+static int walk_hole(struct drm_i915_private *i915,
+ struct i915_address_space *vm,
+ u64 hole_start, u64 hole_end,
+ unsigned long end_time)
+{
+ const u64 hole_size = hole_end - hole_start;
+ const unsigned long max_pages =
+ min_t(u64, ULONG_MAX - 1, hole_size >> PAGE_SHIFT);
+ unsigned long flags;
+ u64 size;
+
+ /* Try binding a single VMA in different positions within the hole */
+
+ flags = PIN_OFFSET_FIXED | PIN_USER;
+ if (i915_is_ggtt(vm))
+ flags |= PIN_GLOBAL;
+
+ for_each_prime_number_from(size, 1, max_pages) {
+ struct drm_i915_gem_object *obj;
+ struct i915_vma *vma;
+ u64 addr;
+ int err = 0;
+
+ obj = fake_dma_object(i915, size << PAGE_SHIFT);
+ if (IS_ERR(obj))
+ break;
+
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto err_put;
+ }
+
+ for (addr = hole_start;
+ addr + obj->base.size < hole_end;
+ addr += obj->base.size) {
+ err = i915_vma_pin(vma, 0, 0, addr | flags);
+ if (err) {
+ pr_err("%s bind failed at %llx + %llx [hole %llx- %llx] with err=%d\n",
+ __func__, addr, vma->size,
+ hole_start, hole_end, err);
+ goto err_close;
+ }
+ i915_vma_unpin(vma);
+
+ if (!drm_mm_node_allocated(&vma->node) ||
+ i915_vma_misplaced(vma, 0, 0, addr | flags)) {
+ pr_err("%s incorrect at %llx + %llx\n",
+ __func__, addr, vma->size);
+ err = -EINVAL;
+ goto err_close;
+ }
+
+ err = i915_vma_unbind(vma);
+ if (err) {
+ pr_err("%s unbind failed at %llx + %llx with err=%d\n",
+ __func__, addr, vma->size, err);
+ goto err_close;
+ }
+
+ GEM_BUG_ON(drm_mm_node_allocated(&vma->node));
+
+ if (igt_timeout(end_time,
+ "%s timed out at %llx\n",
+ __func__, addr)) {
+ err = -EINTR;
+ goto err_close;
+ }
+ }
+
+err_close:
+ if (!i915_vma_is_ggtt(vma))
+ i915_vma_close(vma);
+err_put:
+ i915_gem_object_put(obj);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static int pot_hole(struct drm_i915_private *i915,
+ struct i915_address_space *vm,
+ u64 hole_start, u64 hole_end,
+ unsigned long end_time)
+{
+ struct drm_i915_gem_object *obj;
+ struct i915_vma *vma;
+ unsigned long flags;
+ unsigned int pot;
+ int err = 0;
+
+ flags = PIN_OFFSET_FIXED | PIN_USER;
+ if (i915_is_ggtt(vm))
+ flags |= PIN_GLOBAL;
+
+ obj = i915_gem_object_create_internal(i915, 2 * I915_GTT_PAGE_SIZE);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto err_obj;
+ }
+
+ /* Insert a pair of pages across every pot boundary within the hole */
+ for (pot = fls64(hole_end - 1) - 1;
+ pot > ilog2(2 * I915_GTT_PAGE_SIZE);
+ pot--) {
+ u64 step = BIT_ULL(pot);
+ u64 addr;
+
+ for (addr = round_up(hole_start + I915_GTT_PAGE_SIZE, step) - I915_GTT_PAGE_SIZE;
+ addr <= round_down(hole_end - 2*I915_GTT_PAGE_SIZE, step) - I915_GTT_PAGE_SIZE;
+ addr += step) {
+ err = i915_vma_pin(vma, 0, 0, addr | flags);
+ if (err) {
+ pr_err("%s failed to pin object at %llx in hole [%llx - %llx], with err=%d\n",
+ __func__,
+ addr,
+ hole_start, hole_end,
+ err);
+ goto err;
+ }
+
+ if (!drm_mm_node_allocated(&vma->node) ||
+ i915_vma_misplaced(vma, 0, 0, addr | flags)) {
+ pr_err("%s incorrect at %llx + %llx\n",
+ __func__, addr, vma->size);
+ i915_vma_unpin(vma);
+ err = i915_vma_unbind(vma);
+ err = -EINVAL;
+ goto err;
+ }
+
+ i915_vma_unpin(vma);
+ err = i915_vma_unbind(vma);
+ GEM_BUG_ON(err);
+ }
+
+ if (igt_timeout(end_time,
+ "%s timed out after %d/%d\n",
+ __func__, pot, fls64(hole_end - 1) - 1)) {
+ err = -EINTR;
+ goto err;
+ }
+ }
+
+err:
+ if (!i915_vma_is_ggtt(vma))
+ i915_vma_close(vma);
+err_obj:
+ i915_gem_object_put(obj);
+ return err;
+}
+
+static int drunk_hole(struct drm_i915_private *i915,
+ struct i915_address_space *vm,
+ u64 hole_start, u64 hole_end,
+ unsigned long end_time)
+{
+ I915_RND_STATE(prng);
+ unsigned int size;
+ unsigned long flags;
+
+ flags = PIN_OFFSET_FIXED | PIN_USER;
+ if (i915_is_ggtt(vm))
+ flags |= PIN_GLOBAL;
+
+ /* Keep creating larger objects until one cannot fit into the hole */
+ for (size = 12; (hole_end - hole_start) >> size; size++) {
+ struct drm_i915_gem_object *obj;
+ unsigned int *order, count, n;
+ struct i915_vma *vma;
+ u64 hole_size;
+ int err;
+
+ hole_size = (hole_end - hole_start) >> size;
+ if (hole_size > KMALLOC_MAX_SIZE / sizeof(u32))
+ hole_size = KMALLOC_MAX_SIZE / sizeof(u32);
+ count = hole_size;
+ do {
+ count >>= 1;
+ order = i915_random_order(count, &prng);
+ } while (!order && count);
+ if (!order)
+ break;
+
+ /* Ignore allocation failures (i.e. don't report them as
+ * a test failure) as we are purposefully allocating very
+ * large objects without checking that we have sufficient
+ * memory. We expect to hit -ENOMEM.
+ */
+
+ obj = fake_dma_object(i915, BIT_ULL(size));
+ if (IS_ERR(obj)) {
+ kfree(order);
+ break;
+ }
+
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto err_obj;
+ }
+
+ GEM_BUG_ON(vma->size != BIT_ULL(size));
+
+ for (n = 0; n < count; n++) {
+ u64 addr = hole_start + order[n] * BIT_ULL(size);
+
+ err = i915_vma_pin(vma, 0, 0, addr | flags);
+ if (err) {
+ pr_err("%s failed to pin object at %llx + %llx in hole [%llx - %llx], with err=%d\n",
+ __func__,
+ addr, BIT_ULL(size),
+ hole_start, hole_end,
+ err);
+ goto err;
+ }
+
+ if (!drm_mm_node_allocated(&vma->node) ||
+ i915_vma_misplaced(vma, 0, 0, addr | flags)) {
+ pr_err("%s incorrect at %llx + %llx\n",
+ __func__, addr, BIT_ULL(size));
+ i915_vma_unpin(vma);
+ err = i915_vma_unbind(vma);
+ err = -EINVAL;
+ goto err;
+ }
+
+ i915_vma_unpin(vma);
+ err = i915_vma_unbind(vma);
+ GEM_BUG_ON(err);
+
+ if (igt_timeout(end_time,
+ "%s timed out after %d/%d\n",
+ __func__, n, count)) {
+ err = -EINTR;
+ goto err;
+ }
+ }
+
+err:
+ if (!i915_vma_is_ggtt(vma))
+ i915_vma_close(vma);
+err_obj:
+ i915_gem_object_put(obj);
+ kfree(order);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static int __shrink_hole(struct drm_i915_private *i915,
+ struct i915_address_space *vm,
+ u64 hole_start, u64 hole_end,
+ unsigned long end_time)
+{
+ struct drm_i915_gem_object *obj;
+ unsigned long flags = PIN_OFFSET_FIXED | PIN_USER;
+ unsigned int order = 12;
+ LIST_HEAD(objects);
+ int err = 0;
+ u64 addr;
+
+ /* Keep creating larger objects until one cannot fit into the hole */
+ for (addr = hole_start; addr < hole_end; ) {
+ struct i915_vma *vma;
+ u64 size = BIT_ULL(order++);
+
+ size = min(size, hole_end - addr);
+ obj = fake_dma_object(i915, size);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ break;
+ }
+
+ list_add(&obj->st_link, &objects);
+
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ break;
+ }
+
+ GEM_BUG_ON(vma->size != size);
+
+ err = i915_vma_pin(vma, 0, 0, addr | flags);
+ if (err) {
+ pr_err("%s failed to pin object at %llx + %llx in hole [%llx - %llx], with err=%d\n",
+ __func__, addr, size, hole_start, hole_end, err);
+ break;
+ }
+
+ if (!drm_mm_node_allocated(&vma->node) ||
+ i915_vma_misplaced(vma, 0, 0, addr | flags)) {
+ pr_err("%s incorrect at %llx + %llx\n",
+ __func__, addr, size);
+ i915_vma_unpin(vma);
+ err = i915_vma_unbind(vma);
+ err = -EINVAL;
+ break;
+ }
+
+ i915_vma_unpin(vma);
+ addr += size;
+
+ if (igt_timeout(end_time,
+ "%s timed out at ofset %llx [%llx - %llx]\n",
+ __func__, addr, hole_start, hole_end)) {
+ err = -EINTR;
+ break;
+ }
+ }
+
+ close_object_list(&objects, vm);
+ return err;
+}
+
+static int shrink_hole(struct drm_i915_private *i915,
+ struct i915_address_space *vm,
+ u64 hole_start, u64 hole_end,
+ unsigned long end_time)
+{
+ unsigned long prime;
+ int err;
+
+ vm->fault_attr.probability = 999;
+ atomic_set(&vm->fault_attr.times, -1);
+
+ for_each_prime_number_from(prime, 0, ULONG_MAX - 1) {
+ vm->fault_attr.interval = prime;
+ err = __shrink_hole(i915, vm, hole_start, hole_end, end_time);
+ if (err)
+ break;
+ }
+
+ memset(&vm->fault_attr, 0, sizeof(vm->fault_attr));
+
+ return err;
+}
+
+static int exercise_ppgtt(struct drm_i915_private *dev_priv,
+ int (*func)(struct drm_i915_private *i915,
+ struct i915_address_space *vm,
+ u64 hole_start, u64 hole_end,
+ unsigned long end_time))
+{
+ struct drm_file *file;
+ struct i915_hw_ppgtt *ppgtt;
+ IGT_TIMEOUT(end_time);
+ int err;
+
+ if (!USES_FULL_PPGTT(dev_priv))
+ return 0;
+
+ file = mock_file(dev_priv);
+ if (IS_ERR(file))
+ return PTR_ERR(file);
+
+ mutex_lock(&dev_priv->drm.struct_mutex);
+ ppgtt = i915_ppgtt_create(dev_priv, file->driver_priv, "mock");
+ if (IS_ERR(ppgtt)) {
+ err = PTR_ERR(ppgtt);
+ goto out_unlock;
+ }
+ GEM_BUG_ON(offset_in_page(ppgtt->base.total));
+ GEM_BUG_ON(ppgtt->base.closed);
+
+ err = func(dev_priv, &ppgtt->base, 0, ppgtt->base.total, end_time);
+
+ i915_ppgtt_close(&ppgtt->base);
+ i915_ppgtt_put(ppgtt);
+out_unlock:
+ mutex_unlock(&dev_priv->drm.struct_mutex);
+
+ mock_file_free(dev_priv, file);
+ return err;
+}
+
+static int igt_ppgtt_fill(void *arg)
+{
+ return exercise_ppgtt(arg, fill_hole);
+}
+
+static int igt_ppgtt_walk(void *arg)
+{
+ return exercise_ppgtt(arg, walk_hole);
+}
+
+static int igt_ppgtt_pot(void *arg)
+{
+ return exercise_ppgtt(arg, pot_hole);
+}
+
+static int igt_ppgtt_drunk(void *arg)
+{
+ return exercise_ppgtt(arg, drunk_hole);
+}
+
+static int igt_ppgtt_lowlevel(void *arg)
+{
+ return exercise_ppgtt(arg, lowlevel_hole);
+}
+
+static int igt_ppgtt_shrink(void *arg)
+{
+ return exercise_ppgtt(arg, shrink_hole);
+}
+
+static int sort_holes(void *priv, struct list_head *A, struct list_head *B)
+{
+ struct drm_mm_node *a = list_entry(A, typeof(*a), hole_stack);
+ struct drm_mm_node *b = list_entry(B, typeof(*b), hole_stack);
+
+ if (a->start < b->start)
+ return -1;
+ else
+ return 1;
+}
+
+static int exercise_ggtt(struct drm_i915_private *i915,
+ int (*func)(struct drm_i915_private *i915,
+ struct i915_address_space *vm,
+ u64 hole_start, u64 hole_end,
+ unsigned long end_time))
+{
+ struct i915_ggtt *ggtt = &i915->ggtt;
+ u64 hole_start, hole_end, last = 0;
+ struct drm_mm_node *node;
+ IGT_TIMEOUT(end_time);
+ int err;
+
+ mutex_lock(&i915->drm.struct_mutex);
+restart:
+ list_sort(NULL, &ggtt->base.mm.hole_stack, sort_holes);
+ drm_mm_for_each_hole(node, &ggtt->base.mm, hole_start, hole_end) {
+ if (hole_start < last)
+ continue;
+
+ if (ggtt->base.mm.color_adjust)
+ ggtt->base.mm.color_adjust(node, 0,
+ &hole_start, &hole_end);
+ if (hole_start >= hole_end)
+ continue;
+
+ err = func(i915, &ggtt->base, hole_start, hole_end, end_time);
+ if (err)
+ break;
+
+ /* As we have manipulated the drm_mm, the list may be corrupt */
+ last = hole_end;
+ goto restart;
+ }
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ return err;
+}
+
+static int igt_ggtt_fill(void *arg)
+{
+ return exercise_ggtt(arg, fill_hole);
+}
+
+static int igt_ggtt_walk(void *arg)
+{
+ return exercise_ggtt(arg, walk_hole);
+}
+
+static int igt_ggtt_pot(void *arg)
+{
+ return exercise_ggtt(arg, pot_hole);
+}
+
+static int igt_ggtt_drunk(void *arg)
+{
+ return exercise_ggtt(arg, drunk_hole);
+}
+
+static int igt_ggtt_lowlevel(void *arg)
+{
+ return exercise_ggtt(arg, lowlevel_hole);
+}
+
+static int igt_ggtt_page(void *arg)
+{
+ const unsigned int count = PAGE_SIZE/sizeof(u32);
+ I915_RND_STATE(prng);
+ struct drm_i915_private *i915 = arg;
+ struct i915_ggtt *ggtt = &i915->ggtt;
+ struct drm_i915_gem_object *obj;
+ struct drm_mm_node tmp;
+ unsigned int *order, n;
+ int err;
+
+ mutex_lock(&i915->drm.struct_mutex);
+
+ obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto out_unlock;
+ }
+
+ err = i915_gem_object_pin_pages(obj);
+ if (err)
+ goto out_free;
+
+ memset(&tmp, 0, sizeof(tmp));
+ err = drm_mm_insert_node_in_range(&ggtt->base.mm, &tmp,
+ 1024 * PAGE_SIZE, 0,
+ I915_COLOR_UNEVICTABLE,
+ 0, ggtt->mappable_end,
+ DRM_MM_INSERT_LOW);
+ if (err)
+ goto out_unpin;
+
+ order = i915_random_order(count, &prng);
+ if (!order) {
+ err = -ENOMEM;
+ goto out_remove;
+ }
+
+ for (n = 0; n < count; n++) {
+ u64 offset = tmp.start + order[n] * PAGE_SIZE;
+ u32 __iomem *vaddr;
+
+ ggtt->base.insert_page(&ggtt->base,
+ i915_gem_object_get_dma_address(obj, 0),
+ offset, I915_CACHE_NONE, 0);
+
+ vaddr = io_mapping_map_atomic_wc(&ggtt->mappable, offset);
+ iowrite32(n, vaddr + n);
+ io_mapping_unmap_atomic(vaddr);
+
+ wmb();
+ ggtt->base.clear_range(&ggtt->base, offset, PAGE_SIZE);
+ }
+
+ i915_random_reorder(order, count, &prng);
+ for (n = 0; n < count; n++) {
+ u64 offset = tmp.start + order[n] * PAGE_SIZE;
+ u32 __iomem *vaddr;
+ u32 val;
+
+ ggtt->base.insert_page(&ggtt->base,
+ i915_gem_object_get_dma_address(obj, 0),
+ offset, I915_CACHE_NONE, 0);
+
+ vaddr = io_mapping_map_atomic_wc(&ggtt->mappable, offset);
+ val = ioread32(vaddr + n);
+ io_mapping_unmap_atomic(vaddr);
+
+ ggtt->base.clear_range(&ggtt->base, offset, PAGE_SIZE);
+
+ if (val != n) {
+ pr_err("insert page failed: found %d, expected %d\n",
+ val, n);
+ err = -EINVAL;
+ break;
+ }
+ }
+
+ kfree(order);
+out_remove:
+ drm_mm_remove_node(&tmp);
+out_unpin:
+ i915_gem_object_unpin_pages(obj);
+out_free:
+ i915_gem_object_put(obj);
+out_unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ return err;
+}
+
+static void track_vma_bind(struct i915_vma *vma)
+{
+ struct drm_i915_gem_object *obj = vma->obj;
+
+ obj->bind_count++; /* track for eviction later */
+ __i915_gem_object_pin_pages(obj);
+
+ vma->pages = obj->mm.pages;
+ list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
+}
+
+static int exercise_mock(struct drm_i915_private *i915,
+ int (*func)(struct drm_i915_private *i915,
+ struct i915_address_space *vm,
+ u64 hole_start, u64 hole_end,
+ unsigned long end_time))
+{
+ struct i915_gem_context *ctx;
+ struct i915_hw_ppgtt *ppgtt;
+ IGT_TIMEOUT(end_time);
+ int err;
+
+ ctx = mock_context(i915, "mock");
+ if (!ctx)
+ return -ENOMEM;
+
+ ppgtt = ctx->ppgtt;
+ GEM_BUG_ON(!ppgtt);
+
+ err = func(i915, &ppgtt->base, 0, ppgtt->base.total, end_time);
+
+ mock_context_close(ctx);
+ return err;
+}
+
+static int igt_mock_fill(void *arg)
+{
+ return exercise_mock(arg, fill_hole);
+}
+
+static int igt_mock_walk(void *arg)
+{
+ return exercise_mock(arg, walk_hole);
+}
+
+static int igt_mock_pot(void *arg)
+{
+ return exercise_mock(arg, pot_hole);
+}
+
+static int igt_mock_drunk(void *arg)
+{
+ return exercise_mock(arg, drunk_hole);
+}
+
+static int igt_gtt_reserve(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj, *on;
+ LIST_HEAD(objects);
+ u64 total;
+ int err;
+
+ /* i915_gem_gtt_reserve() tries to reserve the precise range
+ * for the node, and evicts if it has to. So our test checks that
+ * it can give us the requsted space and prevent overlaps.
+ */
+
+ /* Start by filling the GGTT */
+ for (total = 0;
+ total + 2*I915_GTT_PAGE_SIZE <= i915->ggtt.base.total;
+ total += 2*I915_GTT_PAGE_SIZE) {
+ struct i915_vma *vma;
+
+ obj = i915_gem_object_create_internal(i915, 2*PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto out;
+ }
+
+ err = i915_gem_object_pin_pages(obj);
+ if (err) {
+ i915_gem_object_put(obj);
+ goto out;
+ }
+
+ list_add(&obj->st_link, &objects);
+
+ vma = i915_vma_instance(obj, &i915->ggtt.base, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto out;
+ }
+
+ err = i915_gem_gtt_reserve(&i915->ggtt.base, &vma->node,
+ obj->base.size,
+ total,
+ obj->cache_level,
+ 0);
+ if (err) {
+ pr_err("i915_gem_gtt_reserve (pass 1) failed at %llu/%llu with err=%d\n",
+ total, i915->ggtt.base.total, err);
+ goto out;
+ }
+ track_vma_bind(vma);
+
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
+ if (vma->node.start != total ||
+ vma->node.size != 2*I915_GTT_PAGE_SIZE) {
+ pr_err("i915_gem_gtt_reserve (pass 1) placement failed, found (%llx + %llx), expected (%llx + %lx)\n",
+ vma->node.start, vma->node.size,
+ total, 2*I915_GTT_PAGE_SIZE);
+ err = -EINVAL;
+ goto out;
+ }
+ }
+
+ /* Now we start forcing evictions */
+ for (total = I915_GTT_PAGE_SIZE;
+ total + 2*I915_GTT_PAGE_SIZE <= i915->ggtt.base.total;
+ total += 2*I915_GTT_PAGE_SIZE) {
+ struct i915_vma *vma;
+
+ obj = i915_gem_object_create_internal(i915, 2*PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto out;
+ }
+
+ err = i915_gem_object_pin_pages(obj);
+ if (err) {
+ i915_gem_object_put(obj);
+ goto out;
+ }
+
+ list_add(&obj->st_link, &objects);
+
+ vma = i915_vma_instance(obj, &i915->ggtt.base, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto out;
+ }
+
+ err = i915_gem_gtt_reserve(&i915->ggtt.base, &vma->node,
+ obj->base.size,
+ total,
+ obj->cache_level,
+ 0);
+ if (err) {
+ pr_err("i915_gem_gtt_reserve (pass 2) failed at %llu/%llu with err=%d\n",
+ total, i915->ggtt.base.total, err);
+ goto out;
+ }
+ track_vma_bind(vma);
+
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
+ if (vma->node.start != total ||
+ vma->node.size != 2*I915_GTT_PAGE_SIZE) {
+ pr_err("i915_gem_gtt_reserve (pass 2) placement failed, found (%llx + %llx), expected (%llx + %lx)\n",
+ vma->node.start, vma->node.size,
+ total, 2*I915_GTT_PAGE_SIZE);
+ err = -EINVAL;
+ goto out;
+ }
+ }
+
+ /* And then try at random */
+ list_for_each_entry_safe(obj, on, &objects, st_link) {
+ struct i915_vma *vma;
+ u64 offset;
+
+ vma = i915_vma_instance(obj, &i915->ggtt.base, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto out;
+ }
+
+ err = i915_vma_unbind(vma);
+ if (err) {
+ pr_err("i915_vma_unbind failed with err=%d!\n", err);
+ goto out;
+ }
+
+ offset = random_offset(0, i915->ggtt.base.total,
+ 2*I915_GTT_PAGE_SIZE,
+ I915_GTT_MIN_ALIGNMENT);
+
+ err = i915_gem_gtt_reserve(&i915->ggtt.base, &vma->node,
+ obj->base.size,
+ offset,
+ obj->cache_level,
+ 0);
+ if (err) {
+ pr_err("i915_gem_gtt_reserve (pass 3) failed at %llu/%llu with err=%d\n",
+ total, i915->ggtt.base.total, err);
+ goto out;
+ }
+ track_vma_bind(vma);
+
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
+ if (vma->node.start != offset ||
+ vma->node.size != 2*I915_GTT_PAGE_SIZE) {
+ pr_err("i915_gem_gtt_reserve (pass 3) placement failed, found (%llx + %llx), expected (%llx + %lx)\n",
+ vma->node.start, vma->node.size,
+ offset, 2*I915_GTT_PAGE_SIZE);
+ err = -EINVAL;
+ goto out;
+ }
+ }
+
+out:
+ list_for_each_entry_safe(obj, on, &objects, st_link) {
+ i915_gem_object_unpin_pages(obj);
+ i915_gem_object_put(obj);
+ }
+ return err;
+}
+
+static int igt_gtt_insert(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj, *on;
+ struct drm_mm_node tmp = {};
+ const struct invalid_insert {
+ u64 size;
+ u64 alignment;
+ u64 start, end;
+ } invalid_insert[] = {
+ {
+ i915->ggtt.base.total + I915_GTT_PAGE_SIZE, 0,
+ 0, i915->ggtt.base.total,
+ },
+ {
+ 2*I915_GTT_PAGE_SIZE, 0,
+ 0, I915_GTT_PAGE_SIZE,
+ },
+ {
+ -(u64)I915_GTT_PAGE_SIZE, 0,
+ 0, 4*I915_GTT_PAGE_SIZE,
+ },
+ {
+ -(u64)2*I915_GTT_PAGE_SIZE, 2*I915_GTT_PAGE_SIZE,
+ 0, 4*I915_GTT_PAGE_SIZE,
+ },
+ {
+ I915_GTT_PAGE_SIZE, I915_GTT_MIN_ALIGNMENT << 1,
+ I915_GTT_MIN_ALIGNMENT, I915_GTT_MIN_ALIGNMENT << 1,
+ },
+ {}
+ }, *ii;
+ LIST_HEAD(objects);
+ u64 total;
+ int err;
+
+ /* i915_gem_gtt_insert() tries to allocate some free space in the GTT
+ * to the node, evicting if required.
+ */
+
+ /* Check a couple of obviously invalid requests */
+ for (ii = invalid_insert; ii->size; ii++) {
+ err = i915_gem_gtt_insert(&i915->ggtt.base, &tmp,
+ ii->size, ii->alignment,
+ I915_COLOR_UNEVICTABLE,
+ ii->start, ii->end,
+ 0);
+ if (err != -ENOSPC) {
+ pr_err("Invalid i915_gem_gtt_insert(.size=%llx, .alignment=%llx, .start=%llx, .end=%llx) succeeded (err=%d)\n",
+ ii->size, ii->alignment, ii->start, ii->end,
+ err);
+ return -EINVAL;
+ }
+ }
+
+ /* Start by filling the GGTT */
+ for (total = 0;
+ total + I915_GTT_PAGE_SIZE <= i915->ggtt.base.total;
+ total += I915_GTT_PAGE_SIZE) {
+ struct i915_vma *vma;
+
+ obj = i915_gem_object_create_internal(i915, I915_GTT_PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto out;
+ }
+
+ err = i915_gem_object_pin_pages(obj);
+ if (err) {
+ i915_gem_object_put(obj);
+ goto out;
+ }
+
+ list_add(&obj->st_link, &objects);
+
+ vma = i915_vma_instance(obj, &i915->ggtt.base, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto out;
+ }
+
+ err = i915_gem_gtt_insert(&i915->ggtt.base, &vma->node,
+ obj->base.size, 0, obj->cache_level,
+ 0, i915->ggtt.base.total,
+ 0);
+ if (err == -ENOSPC) {
+ /* maxed out the GGTT space */
+ i915_gem_object_put(obj);
+ break;
+ }
+ if (err) {
+ pr_err("i915_gem_gtt_insert (pass 1) failed at %llu/%llu with err=%d\n",
+ total, i915->ggtt.base.total, err);
+ goto out;
+ }
+ track_vma_bind(vma);
+ __i915_vma_pin(vma);
+
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
+ }
+
+ list_for_each_entry(obj, &objects, st_link) {
+ struct i915_vma *vma;
+
+ vma = i915_vma_instance(obj, &i915->ggtt.base, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto out;
+ }
+
+ if (!drm_mm_node_allocated(&vma->node)) {
+ pr_err("VMA was unexpectedly evicted!\n");
+ err = -EINVAL;
+ goto out;
+ }
+
+ __i915_vma_unpin(vma);
+ }
+
+ /* If we then reinsert, we should find the same hole */
+ list_for_each_entry_safe(obj, on, &objects, st_link) {
+ struct i915_vma *vma;
+ u64 offset;
+
+ vma = i915_vma_instance(obj, &i915->ggtt.base, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto out;
+ }
+
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
+ offset = vma->node.start;
+
+ err = i915_vma_unbind(vma);
+ if (err) {
+ pr_err("i915_vma_unbind failed with err=%d!\n", err);
+ goto out;
+ }
+
+ err = i915_gem_gtt_insert(&i915->ggtt.base, &vma->node,
+ obj->base.size, 0, obj->cache_level,
+ 0, i915->ggtt.base.total,
+ 0);
+ if (err) {
+ pr_err("i915_gem_gtt_insert (pass 2) failed at %llu/%llu with err=%d\n",
+ total, i915->ggtt.base.total, err);
+ goto out;
+ }
+ track_vma_bind(vma);
+
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
+ if (vma->node.start != offset) {
+ pr_err("i915_gem_gtt_insert did not return node to its previous location (the only hole), expected address %llx, found %llx\n",
+ offset, vma->node.start);
+ err = -EINVAL;
+ goto out;
+ }
+ }
+
+ /* And then force evictions */
+ for (total = 0;
+ total + 2*I915_GTT_PAGE_SIZE <= i915->ggtt.base.total;
+ total += 2*I915_GTT_PAGE_SIZE) {
+ struct i915_vma *vma;
+
+ obj = i915_gem_object_create_internal(i915, 2*I915_GTT_PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto out;
+ }
+
+ err = i915_gem_object_pin_pages(obj);
+ if (err) {
+ i915_gem_object_put(obj);
+ goto out;
+ }
+
+ list_add(&obj->st_link, &objects);
+
+ vma = i915_vma_instance(obj, &i915->ggtt.base, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto out;
+ }
+
+ err = i915_gem_gtt_insert(&i915->ggtt.base, &vma->node,
+ obj->base.size, 0, obj->cache_level,
+ 0, i915->ggtt.base.total,
+ 0);
+ if (err) {
+ pr_err("i915_gem_gtt_insert (pass 3) failed at %llu/%llu with err=%d\n",
+ total, i915->ggtt.base.total, err);
+ goto out;
+ }
+ track_vma_bind(vma);
+
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
+ }
+
+out:
+ list_for_each_entry_safe(obj, on, &objects, st_link) {
+ i915_gem_object_unpin_pages(obj);
+ i915_gem_object_put(obj);
+ }
+ return err;
+}
+
+int i915_gem_gtt_mock_selftests(void)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_mock_drunk),
+ SUBTEST(igt_mock_walk),
+ SUBTEST(igt_mock_pot),
+ SUBTEST(igt_mock_fill),
+ SUBTEST(igt_gtt_reserve),
+ SUBTEST(igt_gtt_insert),
+ };
+ struct drm_i915_private *i915;
+ int err;
+
+ i915 = mock_gem_device();
+ if (!i915)
+ return -ENOMEM;
+
+ mutex_lock(&i915->drm.struct_mutex);
+ err = i915_subtests(tests, i915);
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ drm_dev_unref(&i915->drm);
+ return err;
+}
+
+int i915_gem_gtt_live_selftests(struct drm_i915_private *i915)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_ppgtt_alloc),
+ SUBTEST(igt_ppgtt_lowlevel),
+ SUBTEST(igt_ppgtt_drunk),
+ SUBTEST(igt_ppgtt_walk),
+ SUBTEST(igt_ppgtt_pot),
+ SUBTEST(igt_ppgtt_fill),
+ SUBTEST(igt_ppgtt_shrink),
+ SUBTEST(igt_ggtt_lowlevel),
+ SUBTEST(igt_ggtt_drunk),
+ SUBTEST(igt_ggtt_walk),
+ SUBTEST(igt_ggtt_pot),
+ SUBTEST(igt_ggtt_fill),
+ SUBTEST(igt_ggtt_page),
+ };
+
+ GEM_BUG_ON(offset_in_page(i915->ggtt.base.total));
+
+ return i915_subtests(tests, i915);
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "../i915_selftest.h"
+
+#include "mock_gem_device.h"
+#include "huge_gem_object.h"
+
+static int igt_gem_object(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj;
+ int err = -ENOMEM;
+
+ /* Basic test to ensure we can create an object */
+
+ obj = i915_gem_object_create(i915, PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ pr_err("i915_gem_object_create failed, err=%d\n", err);
+ goto out;
+ }
+
+ err = 0;
+ i915_gem_object_put(obj);
+out:
+ return err;
+}
+
+static int igt_phys_object(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj;
+ int err;
+
+ /* Create an object and bind it to a contiguous set of physical pages,
+ * i.e. exercise the i915_gem_object_phys API.
+ */
+
+ obj = i915_gem_object_create(i915, PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ pr_err("i915_gem_object_create failed, err=%d\n", err);
+ goto out;
+ }
+
+ mutex_lock(&i915->drm.struct_mutex);
+ err = i915_gem_object_attach_phys(obj, PAGE_SIZE);
+ mutex_unlock(&i915->drm.struct_mutex);
+ if (err) {
+ pr_err("i915_gem_object_attach_phys failed, err=%d\n", err);
+ goto out_obj;
+ }
+
+ if (obj->ops != &i915_gem_phys_ops) {
+ pr_err("i915_gem_object_attach_phys did not create a phys object\n");
+ err = -EINVAL;
+ goto out_obj;
+ }
+
+ if (!atomic_read(&obj->mm.pages_pin_count)) {
+ pr_err("i915_gem_object_attach_phys did not pin its phys pages\n");
+ err = -EINVAL;
+ goto out_obj;
+ }
+
+ /* Make the object dirty so that put_pages must do copy back the data */
+ mutex_lock(&i915->drm.struct_mutex);
+ err = i915_gem_object_set_to_gtt_domain(obj, true);
+ mutex_unlock(&i915->drm.struct_mutex);
+ if (err) {
+ pr_err("i915_gem_object_set_to_gtt_domain failed with err=%d\n",
+ err);
+ goto out_obj;
+ }
+
+out_obj:
+ i915_gem_object_put(obj);
+out:
+ return err;
+}
+
+static int igt_gem_huge(void *arg)
+{
+ const unsigned int nreal = 509; /* just to be awkward */
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj;
+ unsigned int n;
+ int err;
+
+ /* Basic sanitycheck of our huge fake object allocation */
+
+ obj = huge_gem_object(i915,
+ nreal * PAGE_SIZE,
+ i915->ggtt.base.total + PAGE_SIZE);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ err = i915_gem_object_pin_pages(obj);
+ if (err) {
+ pr_err("Failed to allocate %u pages (%lu total), err=%d\n",
+ nreal, obj->base.size / PAGE_SIZE, err);
+ goto out;
+ }
+
+ for (n = 0; n < obj->base.size / PAGE_SIZE; n++) {
+ if (i915_gem_object_get_page(obj, n) !=
+ i915_gem_object_get_page(obj, n % nreal)) {
+ pr_err("Page lookup mismatch at index %u [%u]\n",
+ n, n % nreal);
+ err = -EINVAL;
+ goto out_unpin;
+ }
+ }
+
+out_unpin:
+ i915_gem_object_unpin_pages(obj);
+out:
+ i915_gem_object_put(obj);
+ return err;
+}
+
+struct tile {
+ unsigned int width;
+ unsigned int height;
+ unsigned int stride;
+ unsigned int size;
+ unsigned int tiling;
+ unsigned int swizzle;
+};
+
+static u64 swizzle_bit(unsigned int bit, u64 offset)
+{
+ return (offset & BIT_ULL(bit)) >> (bit - 6);
+}
+
+static u64 tiled_offset(const struct tile *tile, u64 v)
+{
+ u64 x, y;
+
+ if (tile->tiling == I915_TILING_NONE)
+ return v;
+
+ y = div64_u64_rem(v, tile->stride, &x);
+ v = div64_u64_rem(y, tile->height, &y) * tile->stride * tile->height;
+
+ if (tile->tiling == I915_TILING_X) {
+ v += y * tile->width;
+ v += div64_u64_rem(x, tile->width, &x) << tile->size;
+ v += x;
+ } else {
+ const unsigned int ytile_span = 16;
+ const unsigned int ytile_height = 32 * ytile_span;
+
+ v += y * ytile_span;
+ v += div64_u64_rem(x, ytile_span, &x) * ytile_height;
+ v += x;
+ }
+
+ switch (tile->swizzle) {
+ case I915_BIT_6_SWIZZLE_9:
+ v ^= swizzle_bit(9, v);
+ break;
+ case I915_BIT_6_SWIZZLE_9_10:
+ v ^= swizzle_bit(9, v) ^ swizzle_bit(10, v);
+ break;
+ case I915_BIT_6_SWIZZLE_9_11:
+ v ^= swizzle_bit(9, v) ^ swizzle_bit(11, v);
+ break;
+ case I915_BIT_6_SWIZZLE_9_10_11:
+ v ^= swizzle_bit(9, v) ^ swizzle_bit(10, v) ^ swizzle_bit(11, v);
+ break;
+ }
+
+ return v;
+}
+
+static int check_partial_mapping(struct drm_i915_gem_object *obj,
+ const struct tile *tile,
+ unsigned long end_time)
+{
+ const unsigned int nreal = obj->scratch / PAGE_SIZE;
+ const unsigned long npages = obj->base.size / PAGE_SIZE;
+ struct i915_vma *vma;
+ unsigned long page;
+ int err;
+
+ if (igt_timeout(end_time,
+ "%s: timed out before tiling=%d stride=%d\n",
+ __func__, tile->tiling, tile->stride))
+ return -EINTR;
+
+ err = i915_gem_object_set_tiling(obj, tile->tiling, tile->stride);
+ if (err)
+ return err;
+
+ GEM_BUG_ON(i915_gem_object_get_tiling(obj) != tile->tiling);
+ GEM_BUG_ON(i915_gem_object_get_stride(obj) != tile->stride);
+
+ for_each_prime_number_from(page, 1, npages) {
+ struct i915_ggtt_view view =
+ compute_partial_view(obj, page, MIN_CHUNK_PAGES);
+ u32 __iomem *io;
+ struct page *p;
+ unsigned int n;
+ u64 offset;
+ u32 *cpu;
+
+ GEM_BUG_ON(view.partial.size > nreal);
+
+ err = i915_gem_object_set_to_gtt_domain(obj, true);
+ if (err)
+ return err;
+
+ vma = i915_gem_object_ggtt_pin(obj, &view, 0, 0, PIN_MAPPABLE);
+ if (IS_ERR(vma)) {
+ pr_err("Failed to pin partial view: offset=%lu\n",
+ page);
+ return PTR_ERR(vma);
+ }
+
+ n = page - view.partial.offset;
+ GEM_BUG_ON(n >= view.partial.size);
+
+ io = i915_vma_pin_iomap(vma);
+ i915_vma_unpin(vma);
+ if (IS_ERR(io)) {
+ pr_err("Failed to iomap partial view: offset=%lu\n",
+ page);
+ return PTR_ERR(io);
+ }
+
+ err = i915_vma_get_fence(vma);
+ if (err) {
+ pr_err("Failed to get fence for partial view: offset=%lu\n",
+ page);
+ i915_vma_unpin_iomap(vma);
+ return err;
+ }
+
+ iowrite32(page, io + n * PAGE_SIZE/sizeof(*io));
+ i915_vma_unpin_iomap(vma);
+
+ offset = tiled_offset(tile, page << PAGE_SHIFT);
+ if (offset >= obj->base.size)
+ continue;
+
+ i915_gem_object_flush_gtt_write_domain(obj);
+
+ p = i915_gem_object_get_page(obj, offset >> PAGE_SHIFT);
+ cpu = kmap(p) + offset_in_page(offset);
+ drm_clflush_virt_range(cpu, sizeof(*cpu));
+ if (*cpu != (u32)page) {
+ pr_err("Partial view for %lu [%u] (offset=%llu, size=%u [%llu, row size %u], fence=%d, tiling=%d, stride=%d) misalignment, expected write to page (%llu + %u [0x%llx]) of 0x%x, found 0x%x\n",
+ page, n,
+ view.partial.offset,
+ view.partial.size,
+ vma->size >> PAGE_SHIFT,
+ tile_row_pages(obj),
+ vma->fence ? vma->fence->id : -1, tile->tiling, tile->stride,
+ offset >> PAGE_SHIFT,
+ (unsigned int)offset_in_page(offset),
+ offset,
+ (u32)page, *cpu);
+ err = -EINVAL;
+ }
+ *cpu = 0;
+ drm_clflush_virt_range(cpu, sizeof(*cpu));
+ kunmap(p);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static int igt_partial_tiling(void *arg)
+{
+ const unsigned int nreal = 1 << 12; /* largest tile row x2 */
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj;
+ int tiling;
+ int err;
+
+ /* We want to check the page mapping and fencing of a large object
+ * mmapped through the GTT. The object we create is larger than can
+ * possibly be mmaped as a whole, and so we must use partial GGTT vma.
+ * We then check that a write through each partial GGTT vma ends up
+ * in the right set of pages within the object, and with the expected
+ * tiling, which we verify by manual swizzling.
+ */
+
+ obj = huge_gem_object(i915,
+ nreal << PAGE_SHIFT,
+ (1 + next_prime_number(i915->ggtt.base.total >> PAGE_SHIFT)) << PAGE_SHIFT);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ err = i915_gem_object_pin_pages(obj);
+ if (err) {
+ pr_err("Failed to allocate %u pages (%lu total), err=%d\n",
+ nreal, obj->base.size / PAGE_SIZE, err);
+ goto out;
+ }
+
+ mutex_lock(&i915->drm.struct_mutex);
+
+ if (1) {
+ IGT_TIMEOUT(end);
+ struct tile tile;
+
+ tile.height = 1;
+ tile.width = 1;
+ tile.size = 0;
+ tile.stride = 0;
+ tile.swizzle = I915_BIT_6_SWIZZLE_NONE;
+ tile.tiling = I915_TILING_NONE;
+
+ err = check_partial_mapping(obj, &tile, end);
+ if (err && err != -EINTR)
+ goto out_unlock;
+ }
+
+ for (tiling = I915_TILING_X; tiling <= I915_TILING_Y; tiling++) {
+ IGT_TIMEOUT(end);
+ unsigned int max_pitch;
+ unsigned int pitch;
+ struct tile tile;
+
+ tile.tiling = tiling;
+ switch (tiling) {
+ case I915_TILING_X:
+ tile.swizzle = i915->mm.bit_6_swizzle_x;
+ break;
+ case I915_TILING_Y:
+ tile.swizzle = i915->mm.bit_6_swizzle_y;
+ break;
+ }
+
+ if (tile.swizzle == I915_BIT_6_SWIZZLE_UNKNOWN ||
+ tile.swizzle == I915_BIT_6_SWIZZLE_9_10_17)
+ continue;
+
+ if (INTEL_GEN(i915) <= 2) {
+ tile.height = 16;
+ tile.width = 128;
+ tile.size = 11;
+ } else if (tile.tiling == I915_TILING_Y &&
+ HAS_128_BYTE_Y_TILING(i915)) {
+ tile.height = 32;
+ tile.width = 128;
+ tile.size = 12;
+ } else {
+ tile.height = 8;
+ tile.width = 512;
+ tile.size = 12;
+ }
+
+ if (INTEL_GEN(i915) < 4)
+ max_pitch = 8192 / tile.width;
+ else if (INTEL_GEN(i915) < 7)
+ max_pitch = 128 * I965_FENCE_MAX_PITCH_VAL / tile.width;
+ else
+ max_pitch = 128 * GEN7_FENCE_MAX_PITCH_VAL / tile.width;
+
+ for (pitch = max_pitch; pitch; pitch >>= 1) {
+ tile.stride = tile.width * pitch;
+ err = check_partial_mapping(obj, &tile, end);
+ if (err == -EINTR)
+ goto next_tiling;
+ if (err)
+ goto out_unlock;
+
+ if (pitch > 2 && INTEL_GEN(i915) >= 4) {
+ tile.stride = tile.width * (pitch - 1);
+ err = check_partial_mapping(obj, &tile, end);
+ if (err == -EINTR)
+ goto next_tiling;
+ if (err)
+ goto out_unlock;
+ }
+
+ if (pitch < max_pitch && INTEL_GEN(i915) >= 4) {
+ tile.stride = tile.width * (pitch + 1);
+ err = check_partial_mapping(obj, &tile, end);
+ if (err == -EINTR)
+ goto next_tiling;
+ if (err)
+ goto out_unlock;
+ }
+ }
+
+ if (INTEL_GEN(i915) >= 4) {
+ for_each_prime_number(pitch, max_pitch) {
+ tile.stride = tile.width * pitch;
+ err = check_partial_mapping(obj, &tile, end);
+ if (err == -EINTR)
+ goto next_tiling;
+ if (err)
+ goto out_unlock;
+ }
+ }
+
+next_tiling: ;
+ }
+
+out_unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ i915_gem_object_unpin_pages(obj);
+out:
+ i915_gem_object_put(obj);
+ return err;
+}
+
+static int make_obj_busy(struct drm_i915_gem_object *obj)
+{
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+ struct drm_i915_gem_request *rq;
+ struct i915_vma *vma;
+ int err;
+
+ vma = i915_vma_instance(obj, &i915->ggtt.base, NULL);
+ if (IS_ERR(vma))
+ return PTR_ERR(vma);
+
+ err = i915_vma_pin(vma, 0, 0, PIN_USER);
+ if (err)
+ return err;
+
+ rq = i915_gem_request_alloc(i915->engine[RCS], i915->kernel_context);
+ if (IS_ERR(rq)) {
+ i915_vma_unpin(vma);
+ return PTR_ERR(rq);
+ }
+
+ i915_vma_move_to_active(vma, rq, 0);
+ i915_add_request(rq);
+
+ i915_gem_object_set_active_reference(obj);
+ i915_vma_unpin(vma);
+ return 0;
+}
+
+static bool assert_mmap_offset(struct drm_i915_private *i915,
+ unsigned long size,
+ int expected)
+{
+ struct drm_i915_gem_object *obj;
+ int err;
+
+ obj = i915_gem_object_create_internal(i915, size);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ err = i915_gem_object_create_mmap_offset(obj);
+ i915_gem_object_put(obj);
+
+ return err == expected;
+}
+
+static int igt_mmap_offset_exhaustion(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_mm *mm = &i915->drm.vma_offset_manager->vm_addr_space_mm;
+ struct drm_i915_gem_object *obj;
+ struct drm_mm_node resv, *hole;
+ u64 hole_start, hole_end;
+ int loop, err;
+
+ /* Trim the device mmap space to only a page */
+ memset(&resv, 0, sizeof(resv));
+ drm_mm_for_each_hole(hole, mm, hole_start, hole_end) {
+ resv.start = hole_start;
+ resv.size = hole_end - hole_start - 1; /* PAGE_SIZE units */
+ err = drm_mm_reserve_node(mm, &resv);
+ if (err) {
+ pr_err("Failed to trim VMA manager, err=%d\n", err);
+ return err;
+ }
+ break;
+ }
+
+ /* Just fits! */
+ if (!assert_mmap_offset(i915, PAGE_SIZE, 0)) {
+ pr_err("Unable to insert object into single page hole\n");
+ err = -EINVAL;
+ goto out;
+ }
+
+ /* Too large */
+ if (!assert_mmap_offset(i915, 2*PAGE_SIZE, -ENOSPC)) {
+ pr_err("Unexpectedly succeeded in inserting too large object into single page hole\n");
+ err = -EINVAL;
+ goto out;
+ }
+
+ /* Fill the hole, further allocation attempts should then fail */
+ obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto out;
+ }
+
+ err = i915_gem_object_create_mmap_offset(obj);
+ if (err) {
+ pr_err("Unable to insert object into reclaimed hole\n");
+ goto err_obj;
+ }
+
+ if (!assert_mmap_offset(i915, PAGE_SIZE, -ENOSPC)) {
+ pr_err("Unexpectedly succeeded in inserting object into no holes!\n");
+ err = -EINVAL;
+ goto err_obj;
+ }
+
+ i915_gem_object_put(obj);
+
+ /* Now fill with busy dead objects that we expect to reap */
+ for (loop = 0; loop < 3; loop++) {
+ obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto out;
+ }
+
+ mutex_lock(&i915->drm.struct_mutex);
+ err = make_obj_busy(obj);
+ mutex_unlock(&i915->drm.struct_mutex);
+ if (err) {
+ pr_err("[loop %d] Failed to busy the object\n", loop);
+ goto err_obj;
+ }
+
+ GEM_BUG_ON(!i915_gem_object_is_active(obj));
+ err = i915_gem_object_create_mmap_offset(obj);
+ if (err) {
+ pr_err("[loop %d] i915_gem_object_create_mmap_offset failed with err=%d\n",
+ loop, err);
+ goto out;
+ }
+ }
+
+out:
+ drm_mm_remove_node(&resv);
+ return err;
+err_obj:
+ i915_gem_object_put(obj);
+ goto out;
+}
+
+int i915_gem_object_mock_selftests(void)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_gem_object),
+ SUBTEST(igt_phys_object),
+ };
+ struct drm_i915_private *i915;
+ int err;
+
+ i915 = mock_gem_device();
+ if (!i915)
+ return -ENOMEM;
+
+ err = i915_subtests(tests, i915);
+
+ drm_dev_unref(&i915->drm);
+ return err;
+}
+
+int i915_gem_object_live_selftests(struct drm_i915_private *i915)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_gem_huge),
+ SUBTEST(igt_partial_tiling),
+ SUBTEST(igt_mmap_offset_exhaustion),
+ };
+
+ return i915_subtests(tests, i915);
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include <linux/prime_numbers.h>
+
+#include "../i915_selftest.h"
+
+#include "mock_context.h"
+#include "mock_gem_device.h"
+
+static int igt_add_request(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_request *request;
+ int err = -ENOMEM;
+
+ /* Basic preliminary test to create a request and let it loose! */
+
+ mutex_lock(&i915->drm.struct_mutex);
+ request = mock_request(i915->engine[RCS],
+ i915->kernel_context,
+ HZ / 10);
+ if (!request)
+ goto out_unlock;
+
+ i915_add_request(request);
+
+ err = 0;
+out_unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ return err;
+}
+
+static int igt_wait_request(void *arg)
+{
+ const long T = HZ / 4;
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_request *request;
+ int err = -EINVAL;
+
+ /* Submit a request, then wait upon it */
+
+ mutex_lock(&i915->drm.struct_mutex);
+ request = mock_request(i915->engine[RCS], i915->kernel_context, T);
+ if (!request) {
+ err = -ENOMEM;
+ goto out_unlock;
+ }
+
+ if (i915_wait_request(request, I915_WAIT_LOCKED, 0) != -ETIME) {
+ pr_err("request wait (busy query) succeeded (expected timeout before submit!)\n");
+ goto out_unlock;
+ }
+
+ if (i915_wait_request(request, I915_WAIT_LOCKED, T) != -ETIME) {
+ pr_err("request wait succeeded (expected timeout before submit!)\n");
+ goto out_unlock;
+ }
+
+ if (i915_gem_request_completed(request)) {
+ pr_err("request completed before submit!!\n");
+ goto out_unlock;
+ }
+
+ i915_add_request(request);
+
+ if (i915_wait_request(request, I915_WAIT_LOCKED, 0) != -ETIME) {
+ pr_err("request wait (busy query) succeeded (expected timeout after submit!)\n");
+ goto out_unlock;
+ }
+
+ if (i915_gem_request_completed(request)) {
+ pr_err("request completed immediately!\n");
+ goto out_unlock;
+ }
+
+ if (i915_wait_request(request, I915_WAIT_LOCKED, T / 2) != -ETIME) {
+ pr_err("request wait succeeded (expected timeout!)\n");
+ goto out_unlock;
+ }
+
+ if (i915_wait_request(request, I915_WAIT_LOCKED, T) == -ETIME) {
+ pr_err("request wait timed out!\n");
+ goto out_unlock;
+ }
+
+ if (!i915_gem_request_completed(request)) {
+ pr_err("request not complete after waiting!\n");
+ goto out_unlock;
+ }
+
+ if (i915_wait_request(request, I915_WAIT_LOCKED, T) == -ETIME) {
+ pr_err("request wait timed out when already complete!\n");
+ goto out_unlock;
+ }
+
+ err = 0;
+out_unlock:
+ mock_device_flush(i915);
+ mutex_unlock(&i915->drm.struct_mutex);
+ return err;
+}
+
+static int igt_fence_wait(void *arg)
+{
+ const long T = HZ / 4;
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_request *request;
+ int err = -EINVAL;
+
+ /* Submit a request, treat it as a fence and wait upon it */
+
+ mutex_lock(&i915->drm.struct_mutex);
+ request = mock_request(i915->engine[RCS], i915->kernel_context, T);
+ if (!request) {
+ err = -ENOMEM;
+ goto out_locked;
+ }
+ mutex_unlock(&i915->drm.struct_mutex); /* safe as we are single user */
+
+ if (dma_fence_wait_timeout(&request->fence, false, T) != -ETIME) {
+ pr_err("fence wait success before submit (expected timeout)!\n");
+ goto out_device;
+ }
+
+ mutex_lock(&i915->drm.struct_mutex);
+ i915_add_request(request);
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ if (dma_fence_is_signaled(&request->fence)) {
+ pr_err("fence signaled immediately!\n");
+ goto out_device;
+ }
+
+ if (dma_fence_wait_timeout(&request->fence, false, T / 2) != -ETIME) {
+ pr_err("fence wait success after submit (expected timeout)!\n");
+ goto out_device;
+ }
+
+ if (dma_fence_wait_timeout(&request->fence, false, T) <= 0) {
+ pr_err("fence wait timed out (expected success)!\n");
+ goto out_device;
+ }
+
+ if (!dma_fence_is_signaled(&request->fence)) {
+ pr_err("fence unsignaled after waiting!\n");
+ goto out_device;
+ }
+
+ if (dma_fence_wait_timeout(&request->fence, false, T) <= 0) {
+ pr_err("fence wait timed out when complete (expected success)!\n");
+ goto out_device;
+ }
+
+ err = 0;
+out_device:
+ mutex_lock(&i915->drm.struct_mutex);
+out_locked:
+ mock_device_flush(i915);
+ mutex_unlock(&i915->drm.struct_mutex);
+ return err;
+}
+
+static int igt_request_rewind(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_request *request, *vip;
+ struct i915_gem_context *ctx[2];
+ int err = -EINVAL;
+
+ mutex_lock(&i915->drm.struct_mutex);
+ ctx[0] = mock_context(i915, "A");
+ request = mock_request(i915->engine[RCS], ctx[0], 2 * HZ);
+ if (!request) {
+ err = -ENOMEM;
+ goto err_context_0;
+ }
+
+ i915_gem_request_get(request);
+ i915_add_request(request);
+
+ ctx[1] = mock_context(i915, "B");
+ vip = mock_request(i915->engine[RCS], ctx[1], 0);
+ if (!vip) {
+ err = -ENOMEM;
+ goto err_context_1;
+ }
+
+ /* Simulate preemption by manual reordering */
+ if (!mock_cancel_request(request)) {
+ pr_err("failed to cancel request (already executed)!\n");
+ i915_add_request(vip);
+ goto err_context_1;
+ }
+ i915_gem_request_get(vip);
+ i915_add_request(vip);
+ request->engine->submit_request(request);
+
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ if (i915_wait_request(vip, 0, HZ) == -ETIME) {
+ pr_err("timed out waiting for high priority request, vip.seqno=%d, current seqno=%d\n",
+ vip->global_seqno, intel_engine_get_seqno(i915->engine[RCS]));
+ goto err;
+ }
+
+ if (i915_gem_request_completed(request)) {
+ pr_err("low priority request already completed\n");
+ goto err;
+ }
+
+ err = 0;
+err:
+ i915_gem_request_put(vip);
+ mutex_lock(&i915->drm.struct_mutex);
+err_context_1:
+ mock_context_close(ctx[1]);
+ i915_gem_request_put(request);
+err_context_0:
+ mock_context_close(ctx[0]);
+ mock_device_flush(i915);
+ mutex_unlock(&i915->drm.struct_mutex);
+ return err;
+}
+
+int i915_gem_request_mock_selftests(void)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_add_request),
+ SUBTEST(igt_wait_request),
+ SUBTEST(igt_fence_wait),
+ SUBTEST(igt_request_rewind),
+ };
+ struct drm_i915_private *i915;
+ int err;
+
+ i915 = mock_gem_device();
+ if (!i915)
+ return -ENOMEM;
+
+ err = i915_subtests(tests, i915);
+ drm_dev_unref(&i915->drm);
+
+ return err;
+}
+
+struct live_test {
+ struct drm_i915_private *i915;
+ const char *func;
+ const char *name;
+
+ unsigned int reset_count;
+};
+
+static int begin_live_test(struct live_test *t,
+ struct drm_i915_private *i915,
+ const char *func,
+ const char *name)
+{
+ int err;
+
+ t->i915 = i915;
+ t->func = func;
+ t->name = name;
+
+ err = i915_gem_wait_for_idle(i915, I915_WAIT_LOCKED);
+ if (err) {
+ pr_err("%s(%s): failed to idle before, with err=%d!",
+ func, name, err);
+ return err;
+ }
+
+ i915_gem_retire_requests(i915);
+
+ i915->gpu_error.missed_irq_rings = 0;
+ t->reset_count = i915_reset_count(&i915->gpu_error);
+
+ return 0;
+}
+
+static int end_live_test(struct live_test *t)
+{
+ struct drm_i915_private *i915 = t->i915;
+
+ if (wait_for(intel_engines_are_idle(i915), 1)) {
+ pr_err("%s(%s): GPU not idle\n", t->func, t->name);
+ return -EIO;
+ }
+
+ if (t->reset_count != i915_reset_count(&i915->gpu_error)) {
+ pr_err("%s(%s): GPU was reset %d times!\n",
+ t->func, t->name,
+ i915_reset_count(&i915->gpu_error) - t->reset_count);
+ return -EIO;
+ }
+
+ if (i915->gpu_error.missed_irq_rings) {
+ pr_err("%s(%s): Missed interrupts on engines %lx\n",
+ t->func, t->name, i915->gpu_error.missed_irq_rings);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int live_nop_request(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct intel_engine_cs *engine;
+ struct live_test t;
+ unsigned int id;
+ int err;
+
+ /* Submit various sized batches of empty requests, to each engine
+ * (individually), and wait for the batch to complete. We can check
+ * the overhead of submitting requests to the hardware.
+ */
+
+ mutex_lock(&i915->drm.struct_mutex);
+
+ for_each_engine(engine, i915, id) {
+ IGT_TIMEOUT(end_time);
+ struct drm_i915_gem_request *request;
+ unsigned long n, prime;
+ ktime_t times[2] = {};
+
+ err = begin_live_test(&t, i915, __func__, engine->name);
+ if (err)
+ goto out_unlock;
+
+ for_each_prime_number_from(prime, 1, 8192) {
+ times[1] = ktime_get_raw();
+
+ for (n = 0; n < prime; n++) {
+ request = i915_gem_request_alloc(engine,
+ i915->kernel_context);
+ if (IS_ERR(request)) {
+ err = PTR_ERR(request);
+ goto out_unlock;
+ }
+
+ /* This space is left intentionally blank.
+ *
+ * We do not actually want to perform any
+ * action with this request, we just want
+ * to measure the latency in allocation
+ * and submission of our breadcrumbs -
+ * ensuring that the bare request is sufficient
+ * for the system to work (i.e. proper HEAD
+ * tracking of the rings, interrupt handling,
+ * etc). It also gives us the lowest bounds
+ * for latency.
+ */
+
+ i915_add_request(request);
+ }
+ i915_wait_request(request,
+ I915_WAIT_LOCKED,
+ MAX_SCHEDULE_TIMEOUT);
+
+ times[1] = ktime_sub(ktime_get_raw(), times[1]);
+ if (prime == 1)
+ times[0] = times[1];
+
+ if (__igt_timeout(end_time, NULL))
+ break;
+ }
+
+ err = end_live_test(&t);
+ if (err)
+ goto out_unlock;
+
+ pr_info("Request latencies on %s: 1 = %lluns, %lu = %lluns\n",
+ engine->name,
+ ktime_to_ns(times[0]),
+ prime, div64_u64(ktime_to_ns(times[1]), prime));
+ }
+
+out_unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ return err;
+}
+
+static struct i915_vma *empty_batch(struct drm_i915_private *i915)
+{
+ struct drm_i915_gem_object *obj;
+ struct i915_vma *vma;
+ u32 *cmd;
+ int err;
+
+ obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj))
+ return ERR_CAST(obj);
+
+ cmd = i915_gem_object_pin_map(obj, I915_MAP_WB);
+ if (IS_ERR(cmd)) {
+ err = PTR_ERR(cmd);
+ goto err;
+ }
+ *cmd = MI_BATCH_BUFFER_END;
+ i915_gem_object_unpin_map(obj);
+
+ err = i915_gem_object_set_to_gtt_domain(obj, false);
+ if (err)
+ goto err;
+
+ vma = i915_vma_instance(obj, &i915->ggtt.base, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto err;
+ }
+
+ err = i915_vma_pin(vma, 0, 0, PIN_USER | PIN_GLOBAL);
+ if (err)
+ goto err;
+
+ return vma;
+
+err:
+ i915_gem_object_put(obj);
+ return ERR_PTR(err);
+}
+
+static struct drm_i915_gem_request *
+empty_request(struct intel_engine_cs *engine,
+ struct i915_vma *batch)
+{
+ struct drm_i915_gem_request *request;
+ int err;
+
+ request = i915_gem_request_alloc(engine,
+ engine->i915->kernel_context);
+ if (IS_ERR(request))
+ return request;
+
+ err = engine->emit_flush(request, EMIT_INVALIDATE);
+ if (err)
+ goto out_request;
+
+ err = i915_switch_context(request);
+ if (err)
+ goto out_request;
+
+ err = engine->emit_bb_start(request,
+ batch->node.start,
+ batch->node.size,
+ I915_DISPATCH_SECURE);
+ if (err)
+ goto out_request;
+
+out_request:
+ __i915_add_request(request, err == 0);
+ return err ? ERR_PTR(err) : request;
+}
+
+static int live_empty_request(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct intel_engine_cs *engine;
+ struct live_test t;
+ struct i915_vma *batch;
+ unsigned int id;
+ int err = 0;
+
+ /* Submit various sized batches of empty requests, to each engine
+ * (individually), and wait for the batch to complete. We can check
+ * the overhead of submitting requests to the hardware.
+ */
+
+ mutex_lock(&i915->drm.struct_mutex);
+
+ batch = empty_batch(i915);
+ if (IS_ERR(batch)) {
+ err = PTR_ERR(batch);
+ goto out_unlock;
+ }
+
+ for_each_engine(engine, i915, id) {
+ IGT_TIMEOUT(end_time);
+ struct drm_i915_gem_request *request;
+ unsigned long n, prime;
+ ktime_t times[2] = {};
+
+ err = begin_live_test(&t, i915, __func__, engine->name);
+ if (err)
+ goto out_batch;
+
+ /* Warmup / preload */
+ request = empty_request(engine, batch);
+ if (IS_ERR(request)) {
+ err = PTR_ERR(request);
+ goto out_batch;
+ }
+ i915_wait_request(request,
+ I915_WAIT_LOCKED,
+ MAX_SCHEDULE_TIMEOUT);
+
+ for_each_prime_number_from(prime, 1, 8192) {
+ times[1] = ktime_get_raw();
+
+ for (n = 0; n < prime; n++) {
+ request = empty_request(engine, batch);
+ if (IS_ERR(request)) {
+ err = PTR_ERR(request);
+ goto out_batch;
+ }
+ }
+ i915_wait_request(request,
+ I915_WAIT_LOCKED,
+ MAX_SCHEDULE_TIMEOUT);
+
+ times[1] = ktime_sub(ktime_get_raw(), times[1]);
+ if (prime == 1)
+ times[0] = times[1];
+
+ if (__igt_timeout(end_time, NULL))
+ break;
+ }
+
+ err = end_live_test(&t);
+ if (err)
+ goto out_batch;
+
+ pr_info("Batch latencies on %s: 1 = %lluns, %lu = %lluns\n",
+ engine->name,
+ ktime_to_ns(times[0]),
+ prime, div64_u64(ktime_to_ns(times[1]), prime));
+ }
+
+out_batch:
+ i915_vma_unpin(batch);
+ i915_vma_put(batch);
+out_unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ return err;
+}
+
+static struct i915_vma *recursive_batch(struct drm_i915_private *i915)
+{
+ struct i915_gem_context *ctx = i915->kernel_context;
+ struct i915_address_space *vm = ctx->ppgtt ? &ctx->ppgtt->base : &i915->ggtt.base;
+ struct drm_i915_gem_object *obj;
+ const int gen = INTEL_GEN(i915);
+ struct i915_vma *vma;
+ u32 *cmd;
+ int err;
+
+ obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj))
+ return ERR_CAST(obj);
+
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto err;
+ }
+
+ err = i915_vma_pin(vma, 0, 0, PIN_USER);
+ if (err)
+ goto err;
+
+ err = i915_gem_object_set_to_gtt_domain(obj, true);
+ if (err)
+ goto err;
+
+ cmd = i915_gem_object_pin_map(obj, I915_MAP_WC);
+ if (IS_ERR(cmd)) {
+ err = PTR_ERR(cmd);
+ goto err;
+ }
+
+ if (gen >= 8) {
+ *cmd++ = MI_BATCH_BUFFER_START | 1 << 8 | 1;
+ *cmd++ = lower_32_bits(vma->node.start);
+ *cmd++ = upper_32_bits(vma->node.start);
+ } else if (gen >= 6) {
+ *cmd++ = MI_BATCH_BUFFER_START | 1 << 8;
+ *cmd++ = lower_32_bits(vma->node.start);
+ } else if (gen >= 4) {
+ *cmd++ = MI_BATCH_BUFFER_START | MI_BATCH_GTT;
+ *cmd++ = lower_32_bits(vma->node.start);
+ } else {
+ *cmd++ = MI_BATCH_BUFFER_START | MI_BATCH_GTT | 1;
+ *cmd++ = lower_32_bits(vma->node.start);
+ }
+ *cmd++ = MI_BATCH_BUFFER_END; /* terminate early in case of error */
+
+ wmb();
+ i915_gem_object_unpin_map(obj);
+
+ return vma;
+
+err:
+ i915_gem_object_put(obj);
+ return ERR_PTR(err);
+}
+
+static int recursive_batch_resolve(struct i915_vma *batch)
+{
+ u32 *cmd;
+
+ cmd = i915_gem_object_pin_map(batch->obj, I915_MAP_WC);
+ if (IS_ERR(cmd))
+ return PTR_ERR(cmd);
+
+ *cmd = MI_BATCH_BUFFER_END;
+ wmb();
+
+ i915_gem_object_unpin_map(batch->obj);
+
+ return 0;
+}
+
+static int live_all_engines(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct intel_engine_cs *engine;
+ struct drm_i915_gem_request *request[I915_NUM_ENGINES];
+ struct i915_vma *batch;
+ struct live_test t;
+ unsigned int id;
+ int err;
+
+ /* Check we can submit requests to all engines simultaneously. We
+ * send a recursive batch to each engine - checking that we don't
+ * block doing so, and that they don't complete too soon.
+ */
+
+ mutex_lock(&i915->drm.struct_mutex);
+
+ err = begin_live_test(&t, i915, __func__, "");
+ if (err)
+ goto out_unlock;
+
+ batch = recursive_batch(i915);
+ if (IS_ERR(batch)) {
+ err = PTR_ERR(batch);
+ pr_err("%s: Unable to create batch, err=%d\n", __func__, err);
+ goto out_unlock;
+ }
+
+ for_each_engine(engine, i915, id) {
+ request[id] = i915_gem_request_alloc(engine,
+ i915->kernel_context);
+ if (IS_ERR(request[id])) {
+ err = PTR_ERR(request[id]);
+ pr_err("%s: Request allocation failed with err=%d\n",
+ __func__, err);
+ goto out_request;
+ }
+
+ err = engine->emit_flush(request[id], EMIT_INVALIDATE);
+ GEM_BUG_ON(err);
+
+ err = i915_switch_context(request[id]);
+ GEM_BUG_ON(err);
+
+ err = engine->emit_bb_start(request[id],
+ batch->node.start,
+ batch->node.size,
+ 0);
+ GEM_BUG_ON(err);
+ request[id]->batch = batch;
+
+ if (!i915_gem_object_has_active_reference(batch->obj)) {
+ i915_gem_object_get(batch->obj);
+ i915_gem_object_set_active_reference(batch->obj);
+ }
+
+ i915_vma_move_to_active(batch, request[id], 0);
+ i915_gem_request_get(request[id]);
+ i915_add_request(request[id]);
+ }
+
+ for_each_engine(engine, i915, id) {
+ if (i915_gem_request_completed(request[id])) {
+ pr_err("%s(%s): request completed too early!\n",
+ __func__, engine->name);
+ err = -EINVAL;
+ goto out_request;
+ }
+ }
+
+ err = recursive_batch_resolve(batch);
+ if (err) {
+ pr_err("%s: failed to resolve batch, err=%d\n", __func__, err);
+ goto out_request;
+ }
+
+ for_each_engine(engine, i915, id) {
+ long timeout;
+
+ timeout = i915_wait_request(request[id],
+ I915_WAIT_LOCKED,
+ MAX_SCHEDULE_TIMEOUT);
+ if (timeout < 0) {
+ err = timeout;
+ pr_err("%s: error waiting for request on %s, err=%d\n",
+ __func__, engine->name, err);
+ goto out_request;
+ }
+
+ GEM_BUG_ON(!i915_gem_request_completed(request[id]));
+ i915_gem_request_put(request[id]);
+ request[id] = NULL;
+ }
+
+ err = end_live_test(&t);
+
+out_request:
+ for_each_engine(engine, i915, id)
+ if (request[id])
+ i915_gem_request_put(request[id]);
+ i915_vma_unpin(batch);
+ i915_vma_put(batch);
+out_unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ return err;
+}
+
+static int live_sequential_engines(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_request *request[I915_NUM_ENGINES] = {};
+ struct drm_i915_gem_request *prev = NULL;
+ struct intel_engine_cs *engine;
+ struct live_test t;
+ unsigned int id;
+ int err;
+
+ /* Check we can submit requests to all engines sequentially, such
+ * that each successive request waits for the earlier ones. This
+ * tests that we don't execute requests out of order, even though
+ * they are running on independent engines.
+ */
+
+ mutex_lock(&i915->drm.struct_mutex);
+
+ err = begin_live_test(&t, i915, __func__, "");
+ if (err)
+ goto out_unlock;
+
+ for_each_engine(engine, i915, id) {
+ struct i915_vma *batch;
+
+ batch = recursive_batch(i915);
+ if (IS_ERR(batch)) {
+ err = PTR_ERR(batch);
+ pr_err("%s: Unable to create batch for %s, err=%d\n",
+ __func__, engine->name, err);
+ goto out_unlock;
+ }
+
+ request[id] = i915_gem_request_alloc(engine,
+ i915->kernel_context);
+ if (IS_ERR(request[id])) {
+ err = PTR_ERR(request[id]);
+ pr_err("%s: Request allocation failed for %s with err=%d\n",
+ __func__, engine->name, err);
+ goto out_request;
+ }
+
+ if (prev) {
+ err = i915_gem_request_await_dma_fence(request[id],
+ &prev->fence);
+ if (err) {
+ i915_add_request(request[id]);
+ pr_err("%s: Request await failed for %s with err=%d\n",
+ __func__, engine->name, err);
+ goto out_request;
+ }
+ }
+
+ err = engine->emit_flush(request[id], EMIT_INVALIDATE);
+ GEM_BUG_ON(err);
+
+ err = i915_switch_context(request[id]);
+ GEM_BUG_ON(err);
+
+ err = engine->emit_bb_start(request[id],
+ batch->node.start,
+ batch->node.size,
+ 0);
+ GEM_BUG_ON(err);
+ request[id]->batch = batch;
+
+ i915_vma_move_to_active(batch, request[id], 0);
+ i915_gem_object_set_active_reference(batch->obj);
+ i915_vma_get(batch);
+
+ i915_gem_request_get(request[id]);
+ i915_add_request(request[id]);
+
+ prev = request[id];
+ }
+
+ for_each_engine(engine, i915, id) {
+ long timeout;
+
+ if (i915_gem_request_completed(request[id])) {
+ pr_err("%s(%s): request completed too early!\n",
+ __func__, engine->name);
+ err = -EINVAL;
+ goto out_request;
+ }
+
+ err = recursive_batch_resolve(request[id]->batch);
+ if (err) {
+ pr_err("%s: failed to resolve batch, err=%d\n",
+ __func__, err);
+ goto out_request;
+ }
+
+ timeout = i915_wait_request(request[id],
+ I915_WAIT_LOCKED,
+ MAX_SCHEDULE_TIMEOUT);
+ if (timeout < 0) {
+ err = timeout;
+ pr_err("%s: error waiting for request on %s, err=%d\n",
+ __func__, engine->name, err);
+ goto out_request;
+ }
+
+ GEM_BUG_ON(!i915_gem_request_completed(request[id]));
+ }
+
+ err = end_live_test(&t);
+
+out_request:
+ for_each_engine(engine, i915, id) {
+ u32 *cmd;
+
+ if (!request[id])
+ break;
+
+ cmd = i915_gem_object_pin_map(request[id]->batch->obj,
+ I915_MAP_WC);
+ if (!IS_ERR(cmd)) {
+ *cmd = MI_BATCH_BUFFER_END;
+ wmb();
+ i915_gem_object_unpin_map(request[id]->batch->obj);
+ }
+
+ i915_vma_put(request[id]->batch);
+ i915_gem_request_put(request[id]);
+ }
+out_unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ return err;
+}
+
+int i915_gem_request_live_selftests(struct drm_i915_private *i915)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(live_nop_request),
+ SUBTEST(live_all_engines),
+ SUBTEST(live_sequential_engines),
+ SUBTEST(live_empty_request),
+ };
+ return i915_subtests(tests, i915);
+}
--- /dev/null
+/* List each unit test as selftest(name, function)
+ *
+ * The name is used as both an enum and expanded as subtest__name to create
+ * a module parameter. It must be unique and legal for a C identifier.
+ *
+ * The function should be of type int function(void). It may be conditionally
+ * compiled using #if IS_ENABLED(DRM_I915_SELFTEST).
+ *
+ * Tests are executed in order by igt/drv_selftest
+ */
+selftest(sanitycheck, i915_live_sanitycheck) /* keep first (igt selfcheck) */
+selftest(uncore, intel_uncore_live_selftests)
+selftest(requests, i915_gem_request_live_selftests)
+selftest(objects, i915_gem_object_live_selftests)
+selftest(dmabuf, i915_gem_dmabuf_live_selftests)
+selftest(coherency, i915_gem_coherency_live_selftests)
+selftest(gtt, i915_gem_gtt_live_selftests)
+selftest(contexts, i915_gem_context_live_selftests)
+selftest(hangcheck, intel_hangcheck_live_selftests)
--- /dev/null
+/* List each unit test as selftest(name, function)
+ *
+ * The name is used as both an enum and expanded as subtest__name to create
+ * a module parameter. It must be unique and legal for a C identifier.
+ *
+ * The function should be of type int function(void). It may be conditionally
+ * compiled using #if IS_ENABLED(DRM_I915_SELFTEST).
+ *
+ * Tests are executed in order by igt/drv_selftest
+ */
+selftest(sanitycheck, i915_mock_sanitycheck) /* keep first (igt selfcheck) */
+selftest(scatterlist, scatterlist_mock_selftests)
+selftest(uncore, intel_uncore_mock_selftests)
+selftest(breadcrumbs, intel_breadcrumbs_mock_selftests)
+selftest(requests, i915_gem_request_mock_selftests)
+selftest(objects, i915_gem_object_mock_selftests)
+selftest(dmabuf, i915_gem_dmabuf_mock_selftests)
+selftest(vma, i915_vma_mock_selftests)
+selftest(evict, i915_gem_evict_mock_selftests)
+selftest(gtt, i915_gem_gtt_mock_selftests)
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include <linux/bitops.h>
+#include <linux/kernel.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+
+#include "i915_random.h"
+
+static inline u32 i915_prandom_u32_max_state(u32 ep_ro, struct rnd_state *state)
+{
+ return upper_32_bits((u64)prandom_u32_state(state) * ep_ro);
+}
+
+void i915_random_reorder(unsigned int *order, unsigned int count,
+ struct rnd_state *state)
+{
+ unsigned int i, j;
+
+ for (i = 0; i < count; i++) {
+ BUILD_BUG_ON(sizeof(unsigned int) > sizeof(u32));
+ j = i915_prandom_u32_max_state(count, state);
+ swap(order[i], order[j]);
+ }
+}
+
+unsigned int *i915_random_order(unsigned int count, struct rnd_state *state)
+{
+ unsigned int *order, i;
+
+ order = kmalloc_array(count, sizeof(*order), GFP_TEMPORARY);
+ if (!order)
+ return order;
+
+ for (i = 0; i < count; i++)
+ order[i] = i;
+
+ i915_random_reorder(order, count, state);
+ return order;
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __I915_SELFTESTS_RANDOM_H__
+#define __I915_SELFTESTS_RANDOM_H__
+
+#include <linux/random.h>
+
+#include "../i915_selftest.h"
+
+#define I915_RND_STATE_INITIALIZER(x) ({ \
+ struct rnd_state state__; \
+ prandom_seed_state(&state__, (x)); \
+ state__; \
+})
+
+#define I915_RND_STATE(name__) \
+ struct rnd_state name__ = I915_RND_STATE_INITIALIZER(i915_selftest.random_seed)
+
+#define I915_RND_SUBSTATE(name__, parent__) \
+ struct rnd_state name__ = I915_RND_STATE_INITIALIZER(prandom_u32_state(&(parent__)))
+
+unsigned int *i915_random_order(unsigned int count,
+ struct rnd_state *state);
+void i915_random_reorder(unsigned int *order,
+ unsigned int count,
+ struct rnd_state *state);
+
+#endif /* !__I915_SELFTESTS_RANDOM_H__ */
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#include <linux/random.h>
+
+#include "../i915_drv.h"
+#include "../i915_selftest.h"
+
+struct i915_selftest i915_selftest __read_mostly = {
+ .timeout_ms = 1000,
+};
+
+int i915_mock_sanitycheck(void)
+{
+ pr_info(DRIVER_NAME ": %s() - ok!\n", __func__);
+ return 0;
+}
+
+int i915_live_sanitycheck(struct drm_i915_private *i915)
+{
+ pr_info("%s: %s() - ok!\n", i915->drm.driver->name, __func__);
+ return 0;
+}
+
+enum {
+#define selftest(name, func) mock_##name,
+#include "i915_mock_selftests.h"
+#undef selftest
+};
+
+enum {
+#define selftest(name, func) live_##name,
+#include "i915_live_selftests.h"
+#undef selftest
+};
+
+struct selftest {
+ bool enabled;
+ const char *name;
+ union {
+ int (*mock)(void);
+ int (*live)(struct drm_i915_private *);
+ };
+};
+
+#define selftest(n, f) [mock_##n] = { .name = #n, { .mock = f } },
+static struct selftest mock_selftests[] = {
+#include "i915_mock_selftests.h"
+};
+#undef selftest
+
+#define selftest(n, f) [live_##n] = { .name = #n, { .live = f } },
+static struct selftest live_selftests[] = {
+#include "i915_live_selftests.h"
+};
+#undef selftest
+
+/* Embed the line number into the parameter name so that we can order tests */
+#define selftest(n, func) selftest_0(n, func, param(n))
+#define param(n) __PASTE(igt__, __PASTE(__LINE__, __mock_##n))
+#define selftest_0(n, func, id) \
+module_param_named(id, mock_selftests[mock_##n].enabled, bool, 0400);
+#include "i915_mock_selftests.h"
+#undef selftest_0
+#undef param
+
+#define param(n) __PASTE(igt__, __PASTE(__LINE__, __live_##n))
+#define selftest_0(n, func, id) \
+module_param_named(id, live_selftests[live_##n].enabled, bool, 0400);
+#include "i915_live_selftests.h"
+#undef selftest_0
+#undef param
+#undef selftest
+
+static void set_default_test_all(struct selftest *st, unsigned int count)
+{
+ unsigned int i;
+
+ for (i = 0; i < count; i++)
+ if (st[i].enabled)
+ return;
+
+ for (i = 0; i < count; i++)
+ st[i].enabled = true;
+}
+
+static int __run_selftests(const char *name,
+ struct selftest *st,
+ unsigned int count,
+ void *data)
+{
+ int err = 0;
+
+ while (!i915_selftest.random_seed)
+ i915_selftest.random_seed = get_random_int();
+
+ i915_selftest.timeout_jiffies =
+ i915_selftest.timeout_ms ?
+ msecs_to_jiffies_timeout(i915_selftest.timeout_ms) :
+ MAX_SCHEDULE_TIMEOUT;
+
+ set_default_test_all(st, count);
+
+ pr_info(DRIVER_NAME ": Performing %s selftests with st_random_seed=0x%x st_timeout=%u\n",
+ name, i915_selftest.random_seed, i915_selftest.timeout_ms);
+
+ /* Tests are listed in order in i915_*_selftests.h */
+ for (; count--; st++) {
+ if (!st->enabled)
+ continue;
+
+ cond_resched();
+ if (signal_pending(current))
+ return -EINTR;
+
+ pr_debug(DRIVER_NAME ": Running %s\n", st->name);
+ if (data)
+ err = st->live(data);
+ else
+ err = st->mock();
+ if (err == -EINTR && !signal_pending(current))
+ err = 0;
+ if (err)
+ break;
+ }
+
+ if (WARN(err > 0 || err == -ENOTTY,
+ "%s returned %d, conflicting with selftest's magic values!\n",
+ st->name, err))
+ err = -1;
+
+ return err;
+}
+
+#define run_selftests(x, data) \
+ __run_selftests(#x, x##_selftests, ARRAY_SIZE(x##_selftests), data)
+
+int i915_mock_selftests(void)
+{
+ int err;
+
+ if (!i915_selftest.mock)
+ return 0;
+
+ err = run_selftests(mock, NULL);
+ if (err) {
+ i915_selftest.mock = err;
+ return err;
+ }
+
+ if (i915_selftest.mock < 0) {
+ i915_selftest.mock = -ENOTTY;
+ return 1;
+ }
+
+ return 0;
+}
+
+int i915_live_selftests(struct pci_dev *pdev)
+{
+ int err;
+
+ if (!i915_selftest.live)
+ return 0;
+
+ err = run_selftests(live, to_i915(pci_get_drvdata(pdev)));
+ if (err) {
+ i915_selftest.live = err;
+ return err;
+ }
+
+ if (i915_selftest.live < 0) {
+ i915_selftest.live = -ENOTTY;
+ return 1;
+ }
+
+ return 0;
+}
+
+int __i915_subtests(const char *caller,
+ const struct i915_subtest *st,
+ unsigned int count,
+ void *data)
+{
+ int err;
+
+ for (; count--; st++) {
+ cond_resched();
+ if (signal_pending(current))
+ return -EINTR;
+
+ pr_debug(DRIVER_NAME ": Running %s/%s\n", caller, st->name);
+ err = st->func(data);
+ if (err && err != -EINTR) {
+ pr_err(DRIVER_NAME "/%s: %s failed with error %d\n",
+ caller, st->name, err);
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+bool __igt_timeout(unsigned long timeout, const char *fmt, ...)
+{
+ va_list va;
+
+ if (!signal_pending(current)) {
+ cond_resched();
+ if (time_before(jiffies, timeout))
+ return false;
+ }
+
+ if (fmt) {
+ va_start(va, fmt);
+ vprintk(fmt, va);
+ va_end(va);
+ }
+
+ return true;
+}
+
+module_param_named(st_random_seed, i915_selftest.random_seed, uint, 0400);
+module_param_named(st_timeout, i915_selftest.timeout_ms, uint, 0400);
+
+module_param_named_unsafe(mock_selftests, i915_selftest.mock, int, 0400);
+MODULE_PARM_DESC(mock_selftests, "Run selftests before loading, using mock hardware (0:disabled [default], 1:run tests then load driver, -1:run tests then exit module)");
+
+module_param_named_unsafe(live_selftests, i915_selftest.live, int, 0400);
+MODULE_PARM_DESC(live_selftests, "Run selftests after driver initialisation on the live system (0:disabled [default], 1:run tests then continue, -1:run tests then exit module)");
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include <linux/prime_numbers.h>
+
+#include "../i915_selftest.h"
+
+#include "mock_gem_device.h"
+#include "mock_context.h"
+
+static bool assert_vma(struct i915_vma *vma,
+ struct drm_i915_gem_object *obj,
+ struct i915_gem_context *ctx)
+{
+ bool ok = true;
+
+ if (vma->vm != &ctx->ppgtt->base) {
+ pr_err("VMA created with wrong VM\n");
+ ok = false;
+ }
+
+ if (vma->size != obj->base.size) {
+ pr_err("VMA created with wrong size, found %llu, expected %zu\n",
+ vma->size, obj->base.size);
+ ok = false;
+ }
+
+ if (vma->ggtt_view.type != I915_GGTT_VIEW_NORMAL) {
+ pr_err("VMA created with wrong type [%d]\n",
+ vma->ggtt_view.type);
+ ok = false;
+ }
+
+ return ok;
+}
+
+static struct i915_vma *
+checked_vma_instance(struct drm_i915_gem_object *obj,
+ struct i915_address_space *vm,
+ struct i915_ggtt_view *view)
+{
+ struct i915_vma *vma;
+ bool ok = true;
+
+ vma = i915_vma_instance(obj, vm, view);
+ if (IS_ERR(vma))
+ return vma;
+
+ /* Manual checks, will be reinforced by i915_vma_compare! */
+ if (vma->vm != vm) {
+ pr_err("VMA's vm [%p] does not match request [%p]\n",
+ vma->vm, vm);
+ ok = false;
+ }
+
+ if (i915_is_ggtt(vm) != i915_vma_is_ggtt(vma)) {
+ pr_err("VMA ggtt status [%d] does not match parent [%d]\n",
+ i915_vma_is_ggtt(vma), i915_is_ggtt(vm));
+ ok = false;
+ }
+
+ if (i915_vma_compare(vma, vm, view)) {
+ pr_err("i915_vma_compare failed with create parmaters!\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (i915_vma_compare(vma, vma->vm,
+ i915_vma_is_ggtt(vma) ? &vma->ggtt_view : NULL)) {
+ pr_err("i915_vma_compare failed with itself\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (!ok) {
+ pr_err("i915_vma_compare failed to detect the difference!\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ return vma;
+}
+
+static int create_vmas(struct drm_i915_private *i915,
+ struct list_head *objects,
+ struct list_head *contexts)
+{
+ struct drm_i915_gem_object *obj;
+ struct i915_gem_context *ctx;
+ int pinned;
+
+ list_for_each_entry(obj, objects, st_link) {
+ for (pinned = 0; pinned <= 1; pinned++) {
+ list_for_each_entry(ctx, contexts, link) {
+ struct i915_address_space *vm =
+ &ctx->ppgtt->base;
+ struct i915_vma *vma;
+ int err;
+
+ vma = checked_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma))
+ return PTR_ERR(vma);
+
+ if (!assert_vma(vma, obj, ctx)) {
+ pr_err("VMA lookup/create failed\n");
+ return -EINVAL;
+ }
+
+ if (!pinned) {
+ err = i915_vma_pin(vma, 0, 0, PIN_USER);
+ if (err) {
+ pr_err("Failed to pin VMA\n");
+ return err;
+ }
+ } else {
+ i915_vma_unpin(vma);
+ }
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int igt_vma_create(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj, *on;
+ struct i915_gem_context *ctx, *cn;
+ unsigned long num_obj, num_ctx;
+ unsigned long no, nc;
+ IGT_TIMEOUT(end_time);
+ LIST_HEAD(contexts);
+ LIST_HEAD(objects);
+ int err;
+
+ /* Exercise creating many vma amonst many objections, checking the
+ * vma creation and lookup routines.
+ */
+
+ no = 0;
+ for_each_prime_number(num_obj, ULONG_MAX - 1) {
+ for (; no < num_obj; no++) {
+ obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj))
+ goto out;
+
+ list_add(&obj->st_link, &objects);
+ }
+
+ nc = 0;
+ for_each_prime_number(num_ctx, MAX_CONTEXT_HW_ID) {
+ for (; nc < num_ctx; nc++) {
+ ctx = mock_context(i915, "mock");
+ if (!ctx)
+ goto out;
+
+ list_move(&ctx->link, &contexts);
+ }
+
+ err = create_vmas(i915, &objects, &contexts);
+ if (err)
+ goto out;
+
+ if (igt_timeout(end_time,
+ "%s timed out: after %lu objects in %lu contexts\n",
+ __func__, no, nc))
+ goto end;
+ }
+
+ list_for_each_entry_safe(ctx, cn, &contexts, link)
+ mock_context_close(ctx);
+ }
+
+end:
+ /* Final pass to lookup all created contexts */
+ err = create_vmas(i915, &objects, &contexts);
+out:
+ list_for_each_entry_safe(ctx, cn, &contexts, link)
+ mock_context_close(ctx);
+
+ list_for_each_entry_safe(obj, on, &objects, st_link)
+ i915_gem_object_put(obj);
+ return err;
+}
+
+struct pin_mode {
+ u64 size;
+ u64 flags;
+ bool (*assert)(const struct i915_vma *,
+ const struct pin_mode *mode,
+ int result);
+ const char *string;
+};
+
+static bool assert_pin_valid(const struct i915_vma *vma,
+ const struct pin_mode *mode,
+ int result)
+{
+ if (result)
+ return false;
+
+ if (i915_vma_misplaced(vma, mode->size, 0, mode->flags))
+ return false;
+
+ return true;
+}
+
+__maybe_unused
+static bool assert_pin_e2big(const struct i915_vma *vma,
+ const struct pin_mode *mode,
+ int result)
+{
+ return result == -E2BIG;
+}
+
+__maybe_unused
+static bool assert_pin_enospc(const struct i915_vma *vma,
+ const struct pin_mode *mode,
+ int result)
+{
+ return result == -ENOSPC;
+}
+
+__maybe_unused
+static bool assert_pin_einval(const struct i915_vma *vma,
+ const struct pin_mode *mode,
+ int result)
+{
+ return result == -EINVAL;
+}
+
+static int igt_vma_pin1(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ const struct pin_mode modes[] = {
+#define VALID(sz, fl) { .size = (sz), .flags = (fl), .assert = assert_pin_valid, .string = #sz ", " #fl ", (valid) " }
+#define __INVALID(sz, fl, check, eval) { .size = (sz), .flags = (fl), .assert = (check), .string = #sz ", " #fl ", (invalid " #eval ")" }
+#define INVALID(sz, fl) __INVALID(sz, fl, assert_pin_einval, EINVAL)
+#define TOOBIG(sz, fl) __INVALID(sz, fl, assert_pin_e2big, E2BIG)
+#define NOSPACE(sz, fl) __INVALID(sz, fl, assert_pin_enospc, ENOSPC)
+ VALID(0, PIN_GLOBAL),
+ VALID(0, PIN_GLOBAL | PIN_MAPPABLE),
+
+ VALID(0, PIN_GLOBAL | PIN_OFFSET_BIAS | 4096),
+ VALID(0, PIN_GLOBAL | PIN_OFFSET_BIAS | 8192),
+ VALID(0, PIN_GLOBAL | PIN_OFFSET_BIAS | (i915->ggtt.mappable_end - 4096)),
+ VALID(0, PIN_GLOBAL | PIN_MAPPABLE | PIN_OFFSET_BIAS | (i915->ggtt.mappable_end - 4096)),
+ VALID(0, PIN_GLOBAL | PIN_OFFSET_BIAS | (i915->ggtt.base.total - 4096)),
+
+ VALID(0, PIN_GLOBAL | PIN_MAPPABLE | PIN_OFFSET_FIXED | (i915->ggtt.mappable_end - 4096)),
+ INVALID(0, PIN_GLOBAL | PIN_MAPPABLE | PIN_OFFSET_FIXED | i915->ggtt.mappable_end),
+ VALID(0, PIN_GLOBAL | PIN_OFFSET_FIXED | (i915->ggtt.base.total - 4096)),
+ INVALID(0, PIN_GLOBAL | PIN_OFFSET_FIXED | i915->ggtt.base.total),
+ INVALID(0, PIN_GLOBAL | PIN_OFFSET_FIXED | round_down(U64_MAX, PAGE_SIZE)),
+
+ VALID(4096, PIN_GLOBAL),
+ VALID(8192, PIN_GLOBAL),
+ VALID(i915->ggtt.mappable_end - 4096, PIN_GLOBAL | PIN_MAPPABLE),
+ VALID(i915->ggtt.mappable_end, PIN_GLOBAL | PIN_MAPPABLE),
+ TOOBIG(i915->ggtt.mappable_end + 4096, PIN_GLOBAL | PIN_MAPPABLE),
+ VALID(i915->ggtt.base.total - 4096, PIN_GLOBAL),
+ VALID(i915->ggtt.base.total, PIN_GLOBAL),
+ TOOBIG(i915->ggtt.base.total + 4096, PIN_GLOBAL),
+ TOOBIG(round_down(U64_MAX, PAGE_SIZE), PIN_GLOBAL),
+ INVALID(8192, PIN_GLOBAL | PIN_MAPPABLE | PIN_OFFSET_FIXED | (i915->ggtt.mappable_end - 4096)),
+ INVALID(8192, PIN_GLOBAL | PIN_OFFSET_FIXED | (i915->ggtt.base.total - 4096)),
+ INVALID(8192, PIN_GLOBAL | PIN_OFFSET_FIXED | (round_down(U64_MAX, PAGE_SIZE) - 4096)),
+
+ VALID(8192, PIN_GLOBAL | PIN_OFFSET_BIAS | (i915->ggtt.mappable_end - 4096)),
+
+#if !IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM)
+ /* Misusing BIAS is a programming error (it is not controllable
+ * from userspace) so when debugging is enabled, it explodes.
+ * However, the tests are still quite interesting for checking
+ * variable start, end and size.
+ */
+ NOSPACE(0, PIN_GLOBAL | PIN_MAPPABLE | PIN_OFFSET_BIAS | i915->ggtt.mappable_end),
+ NOSPACE(0, PIN_GLOBAL | PIN_OFFSET_BIAS | i915->ggtt.base.total),
+ NOSPACE(8192, PIN_GLOBAL | PIN_MAPPABLE | PIN_OFFSET_BIAS | (i915->ggtt.mappable_end - 4096)),
+ NOSPACE(8192, PIN_GLOBAL | PIN_OFFSET_BIAS | (i915->ggtt.base.total - 4096)),
+#endif
+ { },
+#undef NOSPACE
+#undef TOOBIG
+#undef INVALID
+#undef __INVALID
+#undef VALID
+ }, *m;
+ struct drm_i915_gem_object *obj;
+ struct i915_vma *vma;
+ int err = -EINVAL;
+
+ /* Exercise all the weird and wonderful i915_vma_pin requests,
+ * focusing on error handling of boundary conditions.
+ */
+
+ GEM_BUG_ON(!drm_mm_clean(&i915->ggtt.base.mm));
+
+ obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ vma = checked_vma_instance(obj, &i915->ggtt.base, NULL);
+ if (IS_ERR(vma))
+ goto out;
+
+ for (m = modes; m->assert; m++) {
+ err = i915_vma_pin(vma, m->size, 0, m->flags);
+ if (!m->assert(vma, m, err)) {
+ pr_err("%s to pin single page into GGTT with mode[%d:%s]: size=%llx flags=%llx, err=%d\n",
+ m->assert == assert_pin_valid ? "Failed" : "Unexpectedly succeeded",
+ (int)(m - modes), m->string, m->size, m->flags,
+ err);
+ if (!err)
+ i915_vma_unpin(vma);
+ err = -EINVAL;
+ goto out;
+ }
+
+ if (!err) {
+ i915_vma_unpin(vma);
+ err = i915_vma_unbind(vma);
+ if (err) {
+ pr_err("Failed to unbind single page from GGTT, err=%d\n", err);
+ goto out;
+ }
+ }
+ }
+
+ err = 0;
+out:
+ i915_gem_object_put(obj);
+ return err;
+}
+
+static unsigned long rotated_index(const struct intel_rotation_info *r,
+ unsigned int n,
+ unsigned int x,
+ unsigned int y)
+{
+ return (r->plane[n].stride * (r->plane[n].height - y - 1) +
+ r->plane[n].offset + x);
+}
+
+static struct scatterlist *
+assert_rotated(struct drm_i915_gem_object *obj,
+ const struct intel_rotation_info *r, unsigned int n,
+ struct scatterlist *sg)
+{
+ unsigned int x, y;
+
+ for (x = 0; x < r->plane[n].width; x++) {
+ for (y = 0; y < r->plane[n].height; y++) {
+ unsigned long src_idx;
+ dma_addr_t src;
+
+ if (!sg) {
+ pr_err("Invalid sg table: too short at plane %d, (%d, %d)!\n",
+ n, x, y);
+ return ERR_PTR(-EINVAL);
+ }
+
+ src_idx = rotated_index(r, n, x, y);
+ src = i915_gem_object_get_dma_address(obj, src_idx);
+
+ if (sg_dma_len(sg) != PAGE_SIZE) {
+ pr_err("Invalid sg.length, found %d, expected %lu for rotated page (%d, %d) [src index %lu]\n",
+ sg_dma_len(sg), PAGE_SIZE,
+ x, y, src_idx);
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (sg_dma_address(sg) != src) {
+ pr_err("Invalid address for rotated page (%d, %d) [src index %lu]\n",
+ x, y, src_idx);
+ return ERR_PTR(-EINVAL);
+ }
+
+ sg = sg_next(sg);
+ }
+ }
+
+ return sg;
+}
+
+static unsigned int rotated_size(const struct intel_rotation_plane_info *a,
+ const struct intel_rotation_plane_info *b)
+{
+ return a->width * a->height + b->width * b->height;
+}
+
+static int igt_vma_rotate(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct i915_address_space *vm = &i915->ggtt.base;
+ struct drm_i915_gem_object *obj;
+ const struct intel_rotation_plane_info planes[] = {
+ { .width = 1, .height = 1, .stride = 1 },
+ { .width = 2, .height = 2, .stride = 2 },
+ { .width = 4, .height = 4, .stride = 4 },
+ { .width = 8, .height = 8, .stride = 8 },
+
+ { .width = 3, .height = 5, .stride = 3 },
+ { .width = 3, .height = 5, .stride = 4 },
+ { .width = 3, .height = 5, .stride = 5 },
+
+ { .width = 5, .height = 3, .stride = 5 },
+ { .width = 5, .height = 3, .stride = 7 },
+ { .width = 5, .height = 3, .stride = 9 },
+
+ { .width = 4, .height = 6, .stride = 6 },
+ { .width = 6, .height = 4, .stride = 6 },
+ { }
+ }, *a, *b;
+ const unsigned int max_pages = 64;
+ int err = -ENOMEM;
+
+ /* Create VMA for many different combinations of planes and check
+ * that the page layout within the rotated VMA match our expectations.
+ */
+
+ obj = i915_gem_object_create_internal(i915, max_pages * PAGE_SIZE);
+ if (IS_ERR(obj))
+ goto out;
+
+ for (a = planes; a->width; a++) {
+ for (b = planes + ARRAY_SIZE(planes); b-- != planes; ) {
+ struct i915_ggtt_view view;
+ unsigned int n, max_offset;
+
+ max_offset = max(a->stride * a->height,
+ b->stride * b->height);
+ GEM_BUG_ON(max_offset > max_pages);
+ max_offset = max_pages - max_offset;
+
+ view.type = I915_GGTT_VIEW_ROTATED;
+ view.rotated.plane[0] = *a;
+ view.rotated.plane[1] = *b;
+
+ for_each_prime_number_from(view.rotated.plane[0].offset, 0, max_offset) {
+ for_each_prime_number_from(view.rotated.plane[1].offset, 0, max_offset) {
+ struct scatterlist *sg;
+ struct i915_vma *vma;
+
+ vma = checked_vma_instance(obj, vm, &view);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto out_object;
+ }
+
+ err = i915_vma_pin(vma, 0, 0, PIN_GLOBAL);
+ if (err) {
+ pr_err("Failed to pin VMA, err=%d\n", err);
+ goto out_object;
+ }
+
+ if (vma->size != rotated_size(a, b) * PAGE_SIZE) {
+ pr_err("VMA is wrong size, expected %lu, found %llu\n",
+ PAGE_SIZE * rotated_size(a, b), vma->size);
+ err = -EINVAL;
+ goto out_object;
+ }
+
+ if (vma->pages->nents != rotated_size(a, b)) {
+ pr_err("sg table is wrong sizeo, expected %u, found %u nents\n",
+ rotated_size(a, b), vma->pages->nents);
+ err = -EINVAL;
+ goto out_object;
+ }
+
+ if (vma->node.size < vma->size) {
+ pr_err("VMA binding too small, expected %llu, found %llu\n",
+ vma->size, vma->node.size);
+ err = -EINVAL;
+ goto out_object;
+ }
+
+ if (vma->pages == obj->mm.pages) {
+ pr_err("VMA using unrotated object pages!\n");
+ err = -EINVAL;
+ goto out_object;
+ }
+
+ sg = vma->pages->sgl;
+ for (n = 0; n < ARRAY_SIZE(view.rotated.plane); n++) {
+ sg = assert_rotated(obj, &view.rotated, n, sg);
+ if (IS_ERR(sg)) {
+ pr_err("Inconsistent VMA pages for plane %d: [(%d, %d, %d, %d), (%d, %d, %d, %d)]\n", n,
+ view.rotated.plane[0].width,
+ view.rotated.plane[0].height,
+ view.rotated.plane[0].stride,
+ view.rotated.plane[0].offset,
+ view.rotated.plane[1].width,
+ view.rotated.plane[1].height,
+ view.rotated.plane[1].stride,
+ view.rotated.plane[1].offset);
+ err = -EINVAL;
+ goto out_object;
+ }
+ }
+
+ i915_vma_unpin(vma);
+ }
+ }
+ }
+ }
+
+out_object:
+ i915_gem_object_put(obj);
+out:
+ return err;
+}
+
+static bool assert_partial(struct drm_i915_gem_object *obj,
+ struct i915_vma *vma,
+ unsigned long offset,
+ unsigned long size)
+{
+ struct sgt_iter sgt;
+ dma_addr_t dma;
+
+ for_each_sgt_dma(dma, sgt, vma->pages) {
+ dma_addr_t src;
+
+ if (!size) {
+ pr_err("Partial scattergather list too long\n");
+ return false;
+ }
+
+ src = i915_gem_object_get_dma_address(obj, offset);
+ if (src != dma) {
+ pr_err("DMA mismatch for partial page offset %lu\n",
+ offset);
+ return false;
+ }
+
+ offset++;
+ size--;
+ }
+
+ return true;
+}
+
+static bool assert_pin(struct i915_vma *vma,
+ struct i915_ggtt_view *view,
+ u64 size,
+ const char *name)
+{
+ bool ok = true;
+
+ if (vma->size != size) {
+ pr_err("(%s) VMA is wrong size, expected %llu, found %llu\n",
+ name, size, vma->size);
+ ok = false;
+ }
+
+ if (vma->node.size < vma->size) {
+ pr_err("(%s) VMA binding too small, expected %llu, found %llu\n",
+ name, vma->size, vma->node.size);
+ ok = false;
+ }
+
+ if (view && view->type != I915_GGTT_VIEW_NORMAL) {
+ if (memcmp(&vma->ggtt_view, view, sizeof(*view))) {
+ pr_err("(%s) VMA mismatch upon creation!\n",
+ name);
+ ok = false;
+ }
+
+ if (vma->pages == vma->obj->mm.pages) {
+ pr_err("(%s) VMA using original object pages!\n",
+ name);
+ ok = false;
+ }
+ } else {
+ if (vma->ggtt_view.type != I915_GGTT_VIEW_NORMAL) {
+ pr_err("Not the normal ggtt view! Found %d\n",
+ vma->ggtt_view.type);
+ ok = false;
+ }
+
+ if (vma->pages != vma->obj->mm.pages) {
+ pr_err("VMA not using object pages!\n");
+ ok = false;
+ }
+ }
+
+ return ok;
+}
+
+static int igt_vma_partial(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct i915_address_space *vm = &i915->ggtt.base;
+ const unsigned int npages = 1021; /* prime! */
+ struct drm_i915_gem_object *obj;
+ const struct phase {
+ const char *name;
+ } phases[] = {
+ { "create" },
+ { "lookup" },
+ { },
+ }, *p;
+ unsigned int sz, offset;
+ struct i915_vma *vma;
+ int err = -ENOMEM;
+
+ /* Create lots of different VMA for the object and check that
+ * we are returned the same VMA when we later request the same range.
+ */
+
+ obj = i915_gem_object_create_internal(i915, npages*PAGE_SIZE);
+ if (IS_ERR(obj))
+ goto out;
+
+ for (p = phases; p->name; p++) { /* exercise both create/lookup */
+ unsigned int count, nvma;
+
+ nvma = 0;
+ for_each_prime_number_from(sz, 1, npages) {
+ for_each_prime_number_from(offset, 0, npages - sz) {
+ struct i915_ggtt_view view;
+
+ view.type = I915_GGTT_VIEW_PARTIAL;
+ view.partial.offset = offset;
+ view.partial.size = sz;
+
+ if (sz == npages)
+ view.type = I915_GGTT_VIEW_NORMAL;
+
+ vma = checked_vma_instance(obj, vm, &view);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto out_object;
+ }
+
+ err = i915_vma_pin(vma, 0, 0, PIN_GLOBAL);
+ if (err)
+ goto out_object;
+
+ if (!assert_pin(vma, &view, sz*PAGE_SIZE, p->name)) {
+ pr_err("(%s) Inconsistent partial pinning for (offset=%d, size=%d)\n",
+ p->name, offset, sz);
+ err = -EINVAL;
+ goto out_object;
+ }
+
+ if (!assert_partial(obj, vma, offset, sz)) {
+ pr_err("(%s) Inconsistent partial pages for (offset=%d, size=%d)\n",
+ p->name, offset, sz);
+ err = -EINVAL;
+ goto out_object;
+ }
+
+ i915_vma_unpin(vma);
+ nvma++;
+ }
+ }
+
+ count = 0;
+ list_for_each_entry(vma, &obj->vma_list, obj_link)
+ count++;
+ if (count != nvma) {
+ pr_err("(%s) All partial vma were not recorded on the obj->vma_list: found %u, expected %u\n",
+ p->name, count, nvma);
+ err = -EINVAL;
+ goto out_object;
+ }
+
+ /* Check that we did create the whole object mapping */
+ vma = checked_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto out_object;
+ }
+
+ err = i915_vma_pin(vma, 0, 0, PIN_GLOBAL);
+ if (err)
+ goto out_object;
+
+ if (!assert_pin(vma, NULL, obj->base.size, p->name)) {
+ pr_err("(%s) inconsistent full pin\n", p->name);
+ err = -EINVAL;
+ goto out_object;
+ }
+
+ i915_vma_unpin(vma);
+
+ count = 0;
+ list_for_each_entry(vma, &obj->vma_list, obj_link)
+ count++;
+ if (count != nvma) {
+ pr_err("(%s) allocated an extra full vma!\n", p->name);
+ err = -EINVAL;
+ goto out_object;
+ }
+ }
+
+out_object:
+ i915_gem_object_put(obj);
+out:
+ return err;
+}
+
+int i915_vma_mock_selftests(void)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_vma_create),
+ SUBTEST(igt_vma_pin1),
+ SUBTEST(igt_vma_rotate),
+ SUBTEST(igt_vma_partial),
+ };
+ struct drm_i915_private *i915;
+ int err;
+
+ i915 = mock_gem_device();
+ if (!i915)
+ return -ENOMEM;
+
+ mutex_lock(&i915->drm.struct_mutex);
+ err = i915_subtests(tests, i915);
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ drm_dev_unref(&i915->drm);
+ return err;
+}
+
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "../i915_selftest.h"
+#include "i915_random.h"
+
+#include "mock_gem_device.h"
+#include "mock_engine.h"
+
+static int check_rbtree(struct intel_engine_cs *engine,
+ const unsigned long *bitmap,
+ const struct intel_wait *waiters,
+ const int count)
+{
+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
+ struct rb_node *rb;
+ int n;
+
+ if (&b->irq_wait->node != rb_first(&b->waiters)) {
+ pr_err("First waiter does not match first element of wait-tree\n");
+ return -EINVAL;
+ }
+
+ n = find_first_bit(bitmap, count);
+ for (rb = rb_first(&b->waiters); rb; rb = rb_next(rb)) {
+ struct intel_wait *w = container_of(rb, typeof(*w), node);
+ int idx = w - waiters;
+
+ if (!test_bit(idx, bitmap)) {
+ pr_err("waiter[%d, seqno=%d] removed but still in wait-tree\n",
+ idx, w->seqno);
+ return -EINVAL;
+ }
+
+ if (n != idx) {
+ pr_err("waiter[%d, seqno=%d] does not match expected next element in tree [%d]\n",
+ idx, w->seqno, n);
+ return -EINVAL;
+ }
+
+ n = find_next_bit(bitmap, count, n + 1);
+ }
+
+ return 0;
+}
+
+static int check_completion(struct intel_engine_cs *engine,
+ const unsigned long *bitmap,
+ const struct intel_wait *waiters,
+ const int count)
+{
+ int n;
+
+ for (n = 0; n < count; n++) {
+ if (intel_wait_complete(&waiters[n]) != !!test_bit(n, bitmap))
+ continue;
+
+ pr_err("waiter[%d, seqno=%d] is %s, but expected %s\n",
+ n, waiters[n].seqno,
+ intel_wait_complete(&waiters[n]) ? "complete" : "active",
+ test_bit(n, bitmap) ? "active" : "complete");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int check_rbtree_empty(struct intel_engine_cs *engine)
+{
+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
+
+ if (b->irq_wait) {
+ pr_err("Empty breadcrumbs still has a waiter\n");
+ return -EINVAL;
+ }
+
+ if (!RB_EMPTY_ROOT(&b->waiters)) {
+ pr_err("Empty breadcrumbs, but wait-tree not empty\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int igt_random_insert_remove(void *arg)
+{
+ const u32 seqno_bias = 0x1000;
+ I915_RND_STATE(prng);
+ struct intel_engine_cs *engine = arg;
+ struct intel_wait *waiters;
+ const int count = 4096;
+ unsigned int *order;
+ unsigned long *bitmap;
+ int err = -ENOMEM;
+ int n;
+
+ mock_engine_reset(engine);
+
+ waiters = drm_malloc_gfp(count, sizeof(*waiters), GFP_TEMPORARY);
+ if (!waiters)
+ goto out_engines;
+
+ bitmap = kcalloc(DIV_ROUND_UP(count, BITS_PER_LONG), sizeof(*bitmap),
+ GFP_TEMPORARY);
+ if (!bitmap)
+ goto out_waiters;
+
+ order = i915_random_order(count, &prng);
+ if (!order)
+ goto out_bitmap;
+
+ for (n = 0; n < count; n++)
+ intel_wait_init_for_seqno(&waiters[n], seqno_bias + n);
+
+ err = check_rbtree(engine, bitmap, waiters, count);
+ if (err)
+ goto out_order;
+
+ /* Add and remove waiters into the rbtree in random order. At each
+ * step, we verify that the rbtree is correctly ordered.
+ */
+ for (n = 0; n < count; n++) {
+ int i = order[n];
+
+ intel_engine_add_wait(engine, &waiters[i]);
+ __set_bit(i, bitmap);
+
+ err = check_rbtree(engine, bitmap, waiters, count);
+ if (err)
+ goto out_order;
+ }
+
+ i915_random_reorder(order, count, &prng);
+ for (n = 0; n < count; n++) {
+ int i = order[n];
+
+ intel_engine_remove_wait(engine, &waiters[i]);
+ __clear_bit(i, bitmap);
+
+ err = check_rbtree(engine, bitmap, waiters, count);
+ if (err)
+ goto out_order;
+ }
+
+ err = check_rbtree_empty(engine);
+out_order:
+ kfree(order);
+out_bitmap:
+ kfree(bitmap);
+out_waiters:
+ drm_free_large(waiters);
+out_engines:
+ mock_engine_flush(engine);
+ return err;
+}
+
+static int igt_insert_complete(void *arg)
+{
+ const u32 seqno_bias = 0x1000;
+ struct intel_engine_cs *engine = arg;
+ struct intel_wait *waiters;
+ const int count = 4096;
+ unsigned long *bitmap;
+ int err = -ENOMEM;
+ int n, m;
+
+ mock_engine_reset(engine);
+
+ waiters = drm_malloc_gfp(count, sizeof(*waiters), GFP_TEMPORARY);
+ if (!waiters)
+ goto out_engines;
+
+ bitmap = kcalloc(DIV_ROUND_UP(count, BITS_PER_LONG), sizeof(*bitmap),
+ GFP_TEMPORARY);
+ if (!bitmap)
+ goto out_waiters;
+
+ for (n = 0; n < count; n++) {
+ intel_wait_init_for_seqno(&waiters[n], n + seqno_bias);
+ intel_engine_add_wait(engine, &waiters[n]);
+ __set_bit(n, bitmap);
+ }
+ err = check_rbtree(engine, bitmap, waiters, count);
+ if (err)
+ goto out_bitmap;
+
+ /* On each step, we advance the seqno so that several waiters are then
+ * complete (we increase the seqno by increasingly larger values to
+ * retire more and more waiters at once). All retired waiters should
+ * be woken and removed from the rbtree, and so that we check.
+ */
+ for (n = 0; n < count; n = m) {
+ int seqno = 2 * n;
+
+ GEM_BUG_ON(find_first_bit(bitmap, count) != n);
+
+ if (intel_wait_complete(&waiters[n])) {
+ pr_err("waiter[%d, seqno=%d] completed too early\n",
+ n, waiters[n].seqno);
+ err = -EINVAL;
+ goto out_bitmap;
+ }
+
+ /* complete the following waiters */
+ mock_seqno_advance(engine, seqno + seqno_bias);
+ for (m = n; m <= seqno; m++) {
+ if (m == count)
+ break;
+
+ GEM_BUG_ON(!test_bit(m, bitmap));
+ __clear_bit(m, bitmap);
+ }
+
+ intel_engine_remove_wait(engine, &waiters[n]);
+ RB_CLEAR_NODE(&waiters[n].node);
+
+ err = check_rbtree(engine, bitmap, waiters, count);
+ if (err) {
+ pr_err("rbtree corrupt after seqno advance to %d\n",
+ seqno + seqno_bias);
+ goto out_bitmap;
+ }
+
+ err = check_completion(engine, bitmap, waiters, count);
+ if (err) {
+ pr_err("completions after seqno advance to %d failed\n",
+ seqno + seqno_bias);
+ goto out_bitmap;
+ }
+ }
+
+ err = check_rbtree_empty(engine);
+out_bitmap:
+ kfree(bitmap);
+out_waiters:
+ drm_free_large(waiters);
+out_engines:
+ mock_engine_flush(engine);
+ return err;
+}
+
+struct igt_wakeup {
+ struct task_struct *tsk;
+ atomic_t *ready, *set, *done;
+ struct intel_engine_cs *engine;
+ unsigned long flags;
+#define STOP 0
+#define IDLE 1
+ wait_queue_head_t *wq;
+ u32 seqno;
+};
+
+static int wait_atomic(atomic_t *p)
+{
+ schedule();
+ return 0;
+}
+
+static int wait_atomic_timeout(atomic_t *p)
+{
+ return schedule_timeout(10 * HZ) ? 0 : -ETIMEDOUT;
+}
+
+static bool wait_for_ready(struct igt_wakeup *w)
+{
+ DEFINE_WAIT(ready);
+
+ set_bit(IDLE, &w->flags);
+ if (atomic_dec_and_test(w->done))
+ wake_up_atomic_t(w->done);
+
+ if (test_bit(STOP, &w->flags))
+ goto out;
+
+ for (;;) {
+ prepare_to_wait(w->wq, &ready, TASK_INTERRUPTIBLE);
+ if (atomic_read(w->ready) == 0)
+ break;
+
+ schedule();
+ }
+ finish_wait(w->wq, &ready);
+
+out:
+ clear_bit(IDLE, &w->flags);
+ if (atomic_dec_and_test(w->set))
+ wake_up_atomic_t(w->set);
+
+ return !test_bit(STOP, &w->flags);
+}
+
+static int igt_wakeup_thread(void *arg)
+{
+ struct igt_wakeup *w = arg;
+ struct intel_wait wait;
+
+ while (wait_for_ready(w)) {
+ GEM_BUG_ON(kthread_should_stop());
+
+ intel_wait_init_for_seqno(&wait, w->seqno);
+ intel_engine_add_wait(w->engine, &wait);
+ for (;;) {
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ if (i915_seqno_passed(intel_engine_get_seqno(w->engine),
+ w->seqno))
+ break;
+
+ if (test_bit(STOP, &w->flags)) /* emergency escape */
+ break;
+
+ schedule();
+ }
+ intel_engine_remove_wait(w->engine, &wait);
+ __set_current_state(TASK_RUNNING);
+ }
+
+ return 0;
+}
+
+static void igt_wake_all_sync(atomic_t *ready,
+ atomic_t *set,
+ atomic_t *done,
+ wait_queue_head_t *wq,
+ int count)
+{
+ atomic_set(set, count);
+ atomic_set(ready, 0);
+ wake_up_all(wq);
+
+ wait_on_atomic_t(set, wait_atomic, TASK_UNINTERRUPTIBLE);
+ atomic_set(ready, count);
+ atomic_set(done, count);
+}
+
+static int igt_wakeup(void *arg)
+{
+ I915_RND_STATE(prng);
+ const int state = TASK_UNINTERRUPTIBLE;
+ struct intel_engine_cs *engine = arg;
+ struct igt_wakeup *waiters;
+ DECLARE_WAIT_QUEUE_HEAD_ONSTACK(wq);
+ const int count = 4096;
+ const u32 max_seqno = count / 4;
+ atomic_t ready, set, done;
+ int err = -ENOMEM;
+ int n, step;
+
+ mock_engine_reset(engine);
+
+ waiters = drm_malloc_gfp(count, sizeof(*waiters), GFP_TEMPORARY);
+ if (!waiters)
+ goto out_engines;
+
+ /* Create a large number of threads, each waiting on a random seqno.
+ * Multiple waiters will be waiting for the same seqno.
+ */
+ atomic_set(&ready, count);
+ for (n = 0; n < count; n++) {
+ waiters[n].wq = &wq;
+ waiters[n].ready = &ready;
+ waiters[n].set = &set;
+ waiters[n].done = &done;
+ waiters[n].engine = engine;
+ waiters[n].flags = BIT(IDLE);
+
+ waiters[n].tsk = kthread_run(igt_wakeup_thread, &waiters[n],
+ "i915/igt:%d", n);
+ if (IS_ERR(waiters[n].tsk))
+ goto out_waiters;
+
+ get_task_struct(waiters[n].tsk);
+ }
+
+ for (step = 1; step <= max_seqno; step <<= 1) {
+ u32 seqno;
+
+ /* The waiter threads start paused as we assign them a random
+ * seqno and reset the engine. Once the engine is reset,
+ * we signal that the threads may begin their wait upon their
+ * seqno.
+ */
+ for (n = 0; n < count; n++) {
+ GEM_BUG_ON(!test_bit(IDLE, &waiters[n].flags));
+ waiters[n].seqno =
+ 1 + prandom_u32_state(&prng) % max_seqno;
+ }
+ mock_seqno_advance(engine, 0);
+ igt_wake_all_sync(&ready, &set, &done, &wq, count);
+
+ /* Simulate the GPU doing chunks of work, with one or more
+ * seqno appearing to finish at the same time. A random number
+ * of threads will be waiting upon the update and hopefully be
+ * woken.
+ */
+ for (seqno = 1; seqno <= max_seqno + step; seqno += step) {
+ usleep_range(50, 500);
+ mock_seqno_advance(engine, seqno);
+ }
+ GEM_BUG_ON(intel_engine_get_seqno(engine) < 1 + max_seqno);
+
+ /* With the seqno now beyond any of the waiting threads, they
+ * should all be woken, see that they are complete and signal
+ * that they are ready for the next test. We wait until all
+ * threads are complete and waiting for us (i.e. not a seqno).
+ */
+ err = wait_on_atomic_t(&done, wait_atomic_timeout, state);
+ if (err) {
+ pr_err("Timed out waiting for %d remaining waiters\n",
+ atomic_read(&done));
+ break;
+ }
+
+ err = check_rbtree_empty(engine);
+ if (err)
+ break;
+ }
+
+out_waiters:
+ for (n = 0; n < count; n++) {
+ if (IS_ERR(waiters[n].tsk))
+ break;
+
+ set_bit(STOP, &waiters[n].flags);
+ }
+ mock_seqno_advance(engine, INT_MAX); /* wakeup any broken waiters */
+ igt_wake_all_sync(&ready, &set, &done, &wq, n);
+
+ for (n = 0; n < count; n++) {
+ if (IS_ERR(waiters[n].tsk))
+ break;
+
+ kthread_stop(waiters[n].tsk);
+ put_task_struct(waiters[n].tsk);
+ }
+
+ drm_free_large(waiters);
+out_engines:
+ mock_engine_flush(engine);
+ return err;
+}
+
+int intel_breadcrumbs_mock_selftests(void)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_random_insert_remove),
+ SUBTEST(igt_insert_complete),
+ SUBTEST(igt_wakeup),
+ };
+ struct drm_i915_private *i915;
+ int err;
+
+ i915 = mock_gem_device();
+ if (!i915)
+ return -ENOMEM;
+
+ err = i915_subtests(tests, i915->engine[RCS]);
+ drm_dev_unref(&i915->drm);
+
+ return err;
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "../i915_selftest.h"
+
+struct hang {
+ struct drm_i915_private *i915;
+ struct drm_i915_gem_object *hws;
+ struct drm_i915_gem_object *obj;
+ u32 *seqno;
+ u32 *batch;
+};
+
+static int hang_init(struct hang *h, struct drm_i915_private *i915)
+{
+ void *vaddr;
+ int err;
+
+ memset(h, 0, sizeof(*h));
+ h->i915 = i915;
+
+ h->hws = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(h->hws))
+ return PTR_ERR(h->hws);
+
+ h->obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(h->obj)) {
+ err = PTR_ERR(h->obj);
+ goto err_hws;
+ }
+
+ i915_gem_object_set_cache_level(h->hws, I915_CACHE_LLC);
+ vaddr = i915_gem_object_pin_map(h->hws, I915_MAP_WB);
+ if (IS_ERR(vaddr)) {
+ err = PTR_ERR(vaddr);
+ goto err_obj;
+ }
+ h->seqno = memset(vaddr, 0xff, PAGE_SIZE);
+
+ vaddr = i915_gem_object_pin_map(h->obj,
+ HAS_LLC(i915) ? I915_MAP_WB : I915_MAP_WC);
+ if (IS_ERR(vaddr)) {
+ err = PTR_ERR(vaddr);
+ goto err_unpin_hws;
+ }
+ h->batch = vaddr;
+
+ return 0;
+
+err_unpin_hws:
+ i915_gem_object_unpin_map(h->hws);
+err_obj:
+ i915_gem_object_put(h->obj);
+err_hws:
+ i915_gem_object_put(h->hws);
+ return err;
+}
+
+static u64 hws_address(const struct i915_vma *hws,
+ const struct drm_i915_gem_request *rq)
+{
+ return hws->node.start + offset_in_page(sizeof(u32)*rq->fence.context);
+}
+
+static int emit_recurse_batch(struct hang *h,
+ struct drm_i915_gem_request *rq)
+{
+ struct drm_i915_private *i915 = h->i915;
+ struct i915_address_space *vm = rq->ctx->ppgtt ? &rq->ctx->ppgtt->base : &i915->ggtt.base;
+ struct i915_vma *hws, *vma;
+ unsigned int flags;
+ u32 *batch;
+ int err;
+
+ vma = i915_vma_instance(h->obj, vm, NULL);
+ if (IS_ERR(vma))
+ return PTR_ERR(vma);
+
+ hws = i915_vma_instance(h->hws, vm, NULL);
+ if (IS_ERR(hws))
+ return PTR_ERR(hws);
+
+ err = i915_vma_pin(vma, 0, 0, PIN_USER);
+ if (err)
+ return err;
+
+ err = i915_vma_pin(hws, 0, 0, PIN_USER);
+ if (err)
+ goto unpin_vma;
+
+ err = rq->engine->emit_flush(rq, EMIT_INVALIDATE);
+ if (err)
+ goto unpin_hws;
+
+ err = i915_switch_context(rq);
+ if (err)
+ goto unpin_hws;
+
+ i915_vma_move_to_active(vma, rq, 0);
+ if (!i915_gem_object_has_active_reference(vma->obj)) {
+ i915_gem_object_get(vma->obj);
+ i915_gem_object_set_active_reference(vma->obj);
+ }
+
+ i915_vma_move_to_active(hws, rq, 0);
+ if (!i915_gem_object_has_active_reference(hws->obj)) {
+ i915_gem_object_get(hws->obj);
+ i915_gem_object_set_active_reference(hws->obj);
+ }
+
+ batch = h->batch;
+ if (INTEL_GEN(i915) >= 8) {
+ *batch++ = MI_STORE_DWORD_IMM_GEN4;
+ *batch++ = lower_32_bits(hws_address(hws, rq));
+ *batch++ = upper_32_bits(hws_address(hws, rq));
+ *batch++ = rq->fence.seqno;
+ *batch++ = MI_BATCH_BUFFER_START | 1 << 8 | 1;
+ *batch++ = lower_32_bits(vma->node.start);
+ *batch++ = upper_32_bits(vma->node.start);
+ } else if (INTEL_GEN(i915) >= 6) {
+ *batch++ = MI_STORE_DWORD_IMM_GEN4;
+ *batch++ = 0;
+ *batch++ = lower_32_bits(hws_address(hws, rq));
+ *batch++ = rq->fence.seqno;
+ *batch++ = MI_BATCH_BUFFER_START | 1 << 8;
+ *batch++ = lower_32_bits(vma->node.start);
+ } else if (INTEL_GEN(i915) >= 4) {
+ *batch++ = MI_STORE_DWORD_IMM_GEN4 | 1 << 22;
+ *batch++ = 0;
+ *batch++ = lower_32_bits(hws_address(hws, rq));
+ *batch++ = rq->fence.seqno;
+ *batch++ = MI_BATCH_BUFFER_START | 2 << 6;
+ *batch++ = lower_32_bits(vma->node.start);
+ } else {
+ *batch++ = MI_STORE_DWORD_IMM;
+ *batch++ = lower_32_bits(hws_address(hws, rq));
+ *batch++ = rq->fence.seqno;
+ *batch++ = MI_BATCH_BUFFER_START | 2 << 6 | 1;
+ *batch++ = lower_32_bits(vma->node.start);
+ }
+ *batch++ = MI_BATCH_BUFFER_END; /* not reached */
+
+ flags = 0;
+ if (INTEL_GEN(vm->i915) <= 5)
+ flags |= I915_DISPATCH_SECURE;
+
+ err = rq->engine->emit_bb_start(rq, vma->node.start, PAGE_SIZE, flags);
+
+unpin_hws:
+ i915_vma_unpin(hws);
+unpin_vma:
+ i915_vma_unpin(vma);
+ return err;
+}
+
+static struct drm_i915_gem_request *
+hang_create_request(struct hang *h,
+ struct intel_engine_cs *engine,
+ struct i915_gem_context *ctx)
+{
+ struct drm_i915_gem_request *rq;
+ int err;
+
+ if (i915_gem_object_is_active(h->obj)) {
+ struct drm_i915_gem_object *obj;
+ void *vaddr;
+
+ obj = i915_gem_object_create_internal(h->i915, PAGE_SIZE);
+ if (IS_ERR(obj))
+ return ERR_CAST(obj);
+
+ vaddr = i915_gem_object_pin_map(obj,
+ HAS_LLC(h->i915) ? I915_MAP_WB : I915_MAP_WC);
+ if (IS_ERR(vaddr)) {
+ i915_gem_object_put(obj);
+ return ERR_CAST(vaddr);
+ }
+
+ i915_gem_object_unpin_map(h->obj);
+ i915_gem_object_put(h->obj);
+
+ h->obj = obj;
+ h->batch = vaddr;
+ }
+
+ rq = i915_gem_request_alloc(engine, ctx);
+ if (IS_ERR(rq))
+ return rq;
+
+ err = emit_recurse_batch(h, rq);
+ if (err) {
+ __i915_add_request(rq, false);
+ return ERR_PTR(err);
+ }
+
+ return rq;
+}
+
+static u32 hws_seqno(const struct hang *h,
+ const struct drm_i915_gem_request *rq)
+{
+ return READ_ONCE(h->seqno[rq->fence.context % (PAGE_SIZE/sizeof(u32))]);
+}
+
+static void hang_fini(struct hang *h)
+{
+ *h->batch = MI_BATCH_BUFFER_END;
+ wmb();
+
+ i915_gem_object_unpin_map(h->obj);
+ i915_gem_object_put(h->obj);
+
+ i915_gem_object_unpin_map(h->hws);
+ i915_gem_object_put(h->hws);
+
+ i915_gem_wait_for_idle(h->i915, I915_WAIT_LOCKED);
+ i915_gem_retire_requests(h->i915);
+}
+
+static int igt_hang_sanitycheck(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_request *rq;
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ struct hang h;
+ int err;
+
+ /* Basic check that we can execute our hanging batch */
+
+ if (!igt_can_mi_store_dword_imm(i915))
+ return 0;
+
+ mutex_lock(&i915->drm.struct_mutex);
+ err = hang_init(&h, i915);
+ if (err)
+ goto unlock;
+
+ for_each_engine(engine, i915, id) {
+ long timeout;
+
+ rq = hang_create_request(&h, engine, i915->kernel_context);
+ if (IS_ERR(rq)) {
+ err = PTR_ERR(rq);
+ pr_err("Failed to create request for %s, err=%d\n",
+ engine->name, err);
+ goto fini;
+ }
+
+ i915_gem_request_get(rq);
+
+ *h.batch = MI_BATCH_BUFFER_END;
+ __i915_add_request(rq, true);
+
+ timeout = i915_wait_request(rq,
+ I915_WAIT_LOCKED,
+ MAX_SCHEDULE_TIMEOUT);
+ i915_gem_request_put(rq);
+
+ if (timeout < 0) {
+ err = timeout;
+ pr_err("Wait for request failed on %s, err=%d\n",
+ engine->name, err);
+ goto fini;
+ }
+ }
+
+fini:
+ hang_fini(&h);
+unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ return err;
+}
+
+static int igt_global_reset(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ unsigned int reset_count;
+ int err = 0;
+
+ /* Check that we can issue a global GPU reset */
+
+ set_bit(I915_RESET_BACKOFF, &i915->gpu_error.flags);
+ set_bit(I915_RESET_HANDOFF, &i915->gpu_error.flags);
+
+ mutex_lock(&i915->drm.struct_mutex);
+ reset_count = i915_reset_count(&i915->gpu_error);
+
+ i915_reset(i915);
+
+ if (i915_reset_count(&i915->gpu_error) == reset_count) {
+ pr_err("No GPU reset recorded!\n");
+ err = -EINVAL;
+ }
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ GEM_BUG_ON(test_bit(I915_RESET_HANDOFF, &i915->gpu_error.flags));
+ clear_bit(I915_RESET_BACKOFF, &i915->gpu_error.flags);
+ if (i915_terminally_wedged(&i915->gpu_error))
+ err = -EIO;
+
+ return err;
+}
+
+static u32 fake_hangcheck(struct drm_i915_gem_request *rq)
+{
+ u32 reset_count;
+
+ rq->engine->hangcheck.stalled = true;
+ rq->engine->hangcheck.seqno = intel_engine_get_seqno(rq->engine);
+
+ reset_count = i915_reset_count(&rq->i915->gpu_error);
+
+ set_bit(I915_RESET_HANDOFF, &rq->i915->gpu_error.flags);
+ wake_up_all(&rq->i915->gpu_error.wait_queue);
+
+ return reset_count;
+}
+
+static bool wait_for_hang(struct hang *h, struct drm_i915_gem_request *rq)
+{
+ return !(wait_for_us(i915_seqno_passed(hws_seqno(h, rq),
+ rq->fence.seqno),
+ 10) &&
+ wait_for(i915_seqno_passed(hws_seqno(h, rq),
+ rq->fence.seqno),
+ 1000));
+}
+
+static int igt_wait_reset(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_request *rq;
+ unsigned int reset_count;
+ struct hang h;
+ long timeout;
+ int err;
+
+ /* Check that we detect a stuck waiter and issue a reset */
+
+ set_bit(I915_RESET_BACKOFF, &i915->gpu_error.flags);
+
+ mutex_lock(&i915->drm.struct_mutex);
+ err = hang_init(&h, i915);
+ if (err)
+ goto unlock;
+
+ rq = hang_create_request(&h, i915->engine[RCS], i915->kernel_context);
+ if (IS_ERR(rq)) {
+ err = PTR_ERR(rq);
+ goto fini;
+ }
+
+ i915_gem_request_get(rq);
+ __i915_add_request(rq, true);
+
+ if (!wait_for_hang(&h, rq)) {
+ pr_err("Failed to start request %x\n", rq->fence.seqno);
+ err = -EIO;
+ goto out_rq;
+ }
+
+ reset_count = fake_hangcheck(rq);
+
+ timeout = i915_wait_request(rq, I915_WAIT_LOCKED, 10);
+ if (timeout < 0) {
+ pr_err("i915_wait_request failed on a stuck request: err=%ld\n",
+ timeout);
+ err = timeout;
+ goto out_rq;
+ }
+
+ GEM_BUG_ON(test_bit(I915_RESET_HANDOFF, &i915->gpu_error.flags));
+ if (i915_reset_count(&i915->gpu_error) == reset_count) {
+ pr_err("No GPU reset recorded!\n");
+ err = -EINVAL;
+ goto out_rq;
+ }
+
+out_rq:
+ i915_gem_request_put(rq);
+fini:
+ hang_fini(&h);
+unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ clear_bit(I915_RESET_BACKOFF, &i915->gpu_error.flags);
+
+ if (i915_terminally_wedged(&i915->gpu_error))
+ return -EIO;
+
+ return err;
+}
+
+static int igt_reset_queue(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ struct hang h;
+ int err;
+
+ /* Check that we replay pending requests following a hang */
+
+ if (!igt_can_mi_store_dword_imm(i915))
+ return 0;
+
+ set_bit(I915_RESET_BACKOFF, &i915->gpu_error.flags);
+ mutex_lock(&i915->drm.struct_mutex);
+ err = hang_init(&h, i915);
+ if (err)
+ goto unlock;
+
+ for_each_engine(engine, i915, id) {
+ struct drm_i915_gem_request *prev;
+ IGT_TIMEOUT(end_time);
+ unsigned int count;
+
+ prev = hang_create_request(&h, engine, i915->kernel_context);
+ if (IS_ERR(prev)) {
+ err = PTR_ERR(prev);
+ goto fini;
+ }
+
+ i915_gem_request_get(prev);
+ __i915_add_request(prev, true);
+
+ count = 0;
+ do {
+ struct drm_i915_gem_request *rq;
+ unsigned int reset_count;
+
+ rq = hang_create_request(&h,
+ engine,
+ i915->kernel_context);
+ if (IS_ERR(rq)) {
+ err = PTR_ERR(rq);
+ goto fini;
+ }
+
+ i915_gem_request_get(rq);
+ __i915_add_request(rq, true);
+
+ if (!wait_for_hang(&h, prev)) {
+ pr_err("Failed to start request %x\n",
+ prev->fence.seqno);
+ i915_gem_request_put(rq);
+ i915_gem_request_put(prev);
+ err = -EIO;
+ goto fini;
+ }
+
+ reset_count = fake_hangcheck(prev);
+
+ i915_reset(i915);
+
+ GEM_BUG_ON(test_bit(I915_RESET_HANDOFF,
+ &i915->gpu_error.flags));
+
+ if (prev->fence.error != -EIO) {
+ pr_err("GPU reset not recorded on hanging request [fence.error=%d]!\n",
+ prev->fence.error);
+ i915_gem_request_put(rq);
+ i915_gem_request_put(prev);
+ err = -EINVAL;
+ goto fini;
+ }
+
+ if (rq->fence.error) {
+ pr_err("Fence error status not zero [%d] after unrelated reset\n",
+ rq->fence.error);
+ i915_gem_request_put(rq);
+ i915_gem_request_put(prev);
+ err = -EINVAL;
+ goto fini;
+ }
+
+ if (i915_reset_count(&i915->gpu_error) == reset_count) {
+ pr_err("No GPU reset recorded!\n");
+ i915_gem_request_put(rq);
+ i915_gem_request_put(prev);
+ err = -EINVAL;
+ goto fini;
+ }
+
+ i915_gem_request_put(prev);
+ prev = rq;
+ count++;
+ } while (time_before(jiffies, end_time));
+ pr_info("%s: Completed %d resets\n", engine->name, count);
+
+ *h.batch = MI_BATCH_BUFFER_END;
+ wmb();
+
+ i915_gem_request_put(prev);
+ }
+
+fini:
+ hang_fini(&h);
+unlock:
+ mutex_unlock(&i915->drm.struct_mutex);
+ clear_bit(I915_RESET_BACKOFF, &i915->gpu_error.flags);
+
+ if (i915_terminally_wedged(&i915->gpu_error))
+ return -EIO;
+
+ return err;
+}
+
+int intel_hangcheck_live_selftests(struct drm_i915_private *i915)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_hang_sanitycheck),
+ SUBTEST(igt_global_reset),
+ SUBTEST(igt_wait_reset),
+ SUBTEST(igt_reset_queue),
+ };
+
+ if (!intel_has_gpu_reset(i915))
+ return 0;
+
+ return i915_subtests(tests, i915);
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "../i915_selftest.h"
+
+static int intel_fw_table_check(const struct intel_forcewake_range *ranges,
+ unsigned int num_ranges,
+ bool is_watertight)
+{
+ unsigned int i;
+ s32 prev;
+
+ for (i = 0, prev = -1; i < num_ranges; i++, ranges++) {
+ /* Check that the table is watertight */
+ if (is_watertight && (prev + 1) != (s32)ranges->start) {
+ pr_err("%s: entry[%d]:(%x, %x) is not watertight to previous (%x)\n",
+ __func__, i, ranges->start, ranges->end, prev);
+ return -EINVAL;
+ }
+
+ /* Check that the table never goes backwards */
+ if (prev >= (s32)ranges->start) {
+ pr_err("%s: entry[%d]:(%x, %x) is less than the previous (%x)\n",
+ __func__, i, ranges->start, ranges->end, prev);
+ return -EINVAL;
+ }
+
+ /* Check that the entry is valid */
+ if (ranges->start >= ranges->end) {
+ pr_err("%s: entry[%d]:(%x, %x) has negative length\n",
+ __func__, i, ranges->start, ranges->end);
+ return -EINVAL;
+ }
+
+ prev = ranges->end;
+ }
+
+ return 0;
+}
+
+static int intel_shadow_table_check(void)
+{
+ const i915_reg_t *reg = gen8_shadowed_regs;
+ unsigned int i;
+ s32 prev;
+
+ for (i = 0, prev = -1; i < ARRAY_SIZE(gen8_shadowed_regs); i++, reg++) {
+ u32 offset = i915_mmio_reg_offset(*reg);
+
+ if (prev >= (s32)offset) {
+ pr_err("%s: entry[%d]:(%x) is before previous (%x)\n",
+ __func__, i, offset, prev);
+ return -EINVAL;
+ }
+
+ prev = offset;
+ }
+
+ return 0;
+}
+
+int intel_uncore_mock_selftests(void)
+{
+ struct {
+ const struct intel_forcewake_range *ranges;
+ unsigned int num_ranges;
+ bool is_watertight;
+ } fw[] = {
+ { __vlv_fw_ranges, ARRAY_SIZE(__vlv_fw_ranges), false },
+ { __chv_fw_ranges, ARRAY_SIZE(__chv_fw_ranges), false },
+ { __gen9_fw_ranges, ARRAY_SIZE(__gen9_fw_ranges), true },
+ };
+ int err, i;
+
+ for (i = 0; i < ARRAY_SIZE(fw); i++) {
+ err = intel_fw_table_check(fw[i].ranges,
+ fw[i].num_ranges,
+ fw[i].is_watertight);
+ if (err)
+ return err;
+ }
+
+ err = intel_shadow_table_check();
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int intel_uncore_check_forcewake_domains(struct drm_i915_private *dev_priv)
+{
+#define FW_RANGE 0x40000
+ unsigned long *valid;
+ u32 offset;
+ int err;
+
+ if (!HAS_FPGA_DBG_UNCLAIMED(dev_priv) &&
+ !IS_VALLEYVIEW(dev_priv) &&
+ !IS_CHERRYVIEW(dev_priv))
+ return 0;
+
+ if (IS_VALLEYVIEW(dev_priv)) /* XXX system lockup! */
+ return 0;
+
+ if (IS_BROADWELL(dev_priv)) /* XXX random GPU hang afterwards! */
+ return 0;
+
+ valid = kzalloc(BITS_TO_LONGS(FW_RANGE) * sizeof(*valid),
+ GFP_TEMPORARY);
+ if (!valid)
+ return -ENOMEM;
+
+ intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+
+ check_for_unclaimed_mmio(dev_priv);
+ for (offset = 0; offset < FW_RANGE; offset += 4) {
+ i915_reg_t reg = { offset };
+
+ (void)I915_READ_FW(reg);
+ if (!check_for_unclaimed_mmio(dev_priv))
+ set_bit(offset, valid);
+ }
+
+ intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
+
+ err = 0;
+ for_each_set_bit(offset, valid, FW_RANGE) {
+ i915_reg_t reg = { offset };
+
+ intel_uncore_forcewake_reset(dev_priv, false);
+ check_for_unclaimed_mmio(dev_priv);
+
+ (void)I915_READ(reg);
+ if (check_for_unclaimed_mmio(dev_priv)) {
+ pr_err("Unclaimed mmio read to register 0x%04x\n",
+ offset);
+ err = -EINVAL;
+ }
+ }
+
+ kfree(valid);
+ return err;
+}
+
+int intel_uncore_live_selftests(struct drm_i915_private *i915)
+{
+ int err;
+
+ /* Confirm the table we load is still valid */
+ err = intel_fw_table_check(i915->uncore.fw_domains_table,
+ i915->uncore.fw_domains_table_entries,
+ INTEL_GEN(i915) >= 9);
+ if (err)
+ return err;
+
+ err = intel_uncore_check_forcewake_domains(i915);
+ if (err)
+ return err;
+
+ return 0;
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "mock_context.h"
+#include "mock_gtt.h"
+
+struct i915_gem_context *
+mock_context(struct drm_i915_private *i915,
+ const char *name)
+{
+ struct i915_gem_context *ctx;
+ int ret;
+
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return NULL;
+
+ kref_init(&ctx->ref);
+ INIT_LIST_HEAD(&ctx->link);
+ ctx->i915 = i915;
+
+ ret = ida_simple_get(&i915->context_hw_ida,
+ 0, MAX_CONTEXT_HW_ID, GFP_KERNEL);
+ if (ret < 0)
+ goto err_free;
+ ctx->hw_id = ret;
+
+ if (name) {
+ ctx->name = kstrdup(name, GFP_KERNEL);
+ if (!ctx->name)
+ goto err_put;
+
+ ctx->ppgtt = mock_ppgtt(i915, name);
+ if (!ctx->ppgtt)
+ goto err_put;
+ }
+
+ return ctx;
+
+err_free:
+ kfree(ctx);
+ return NULL;
+
+err_put:
+ i915_gem_context_set_closed(ctx);
+ i915_gem_context_put(ctx);
+ return NULL;
+}
+
+void mock_context_close(struct i915_gem_context *ctx)
+{
+ i915_gem_context_set_closed(ctx);
+
+ i915_ppgtt_close(&ctx->ppgtt->base);
+
+ i915_gem_context_put(ctx);
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __MOCK_CONTEXT_H
+#define __MOCK_CONTEXT_H
+
+struct i915_gem_context *
+mock_context(struct drm_i915_private *i915,
+ const char *name);
+
+void mock_context_close(struct i915_gem_context *ctx);
+
+#endif /* !__MOCK_CONTEXT_H */
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "mock_dmabuf.h"
+
+static struct sg_table *mock_map_dma_buf(struct dma_buf_attachment *attachment,
+ enum dma_data_direction dir)
+{
+ struct mock_dmabuf *mock = to_mock(attachment->dmabuf);
+ struct sg_table *st;
+ struct scatterlist *sg;
+ int i, err;
+
+ st = kmalloc(sizeof(*st), GFP_KERNEL);
+ if (!st)
+ return ERR_PTR(-ENOMEM);
+
+ err = sg_alloc_table(st, mock->npages, GFP_KERNEL);
+ if (err)
+ goto err_free;
+
+ sg = st->sgl;
+ for (i = 0; i < mock->npages; i++) {
+ sg_set_page(sg, mock->pages[i], PAGE_SIZE, 0);
+ sg = sg_next(sg);
+ }
+
+ if (!dma_map_sg(attachment->dev, st->sgl, st->nents, dir)) {
+ err = -ENOMEM;
+ goto err_st;
+ }
+
+ return st;
+
+err_st:
+ sg_free_table(st);
+err_free:
+ kfree(st);
+ return ERR_PTR(err);
+}
+
+static void mock_unmap_dma_buf(struct dma_buf_attachment *attachment,
+ struct sg_table *st,
+ enum dma_data_direction dir)
+{
+ dma_unmap_sg(attachment->dev, st->sgl, st->nents, dir);
+ sg_free_table(st);
+ kfree(st);
+}
+
+static void mock_dmabuf_release(struct dma_buf *dma_buf)
+{
+ struct mock_dmabuf *mock = to_mock(dma_buf);
+ int i;
+
+ for (i = 0; i < mock->npages; i++)
+ put_page(mock->pages[i]);
+
+ kfree(mock);
+}
+
+static void *mock_dmabuf_vmap(struct dma_buf *dma_buf)
+{
+ struct mock_dmabuf *mock = to_mock(dma_buf);
+
+ return vm_map_ram(mock->pages, mock->npages, 0, PAGE_KERNEL);
+}
+
+static void mock_dmabuf_vunmap(struct dma_buf *dma_buf, void *vaddr)
+{
+ struct mock_dmabuf *mock = to_mock(dma_buf);
+
+ vm_unmap_ram(vaddr, mock->npages);
+}
+
+static void *mock_dmabuf_kmap_atomic(struct dma_buf *dma_buf, unsigned long page_num)
+{
+ struct mock_dmabuf *mock = to_mock(dma_buf);
+
+ return kmap_atomic(mock->pages[page_num]);
+}
+
+static void mock_dmabuf_kunmap_atomic(struct dma_buf *dma_buf, unsigned long page_num, void *addr)
+{
+ kunmap_atomic(addr);
+}
+
+static void *mock_dmabuf_kmap(struct dma_buf *dma_buf, unsigned long page_num)
+{
+ struct mock_dmabuf *mock = to_mock(dma_buf);
+
+ return kmap(mock->pages[page_num]);
+}
+
+static void mock_dmabuf_kunmap(struct dma_buf *dma_buf, unsigned long page_num, void *addr)
+{
+ struct mock_dmabuf *mock = to_mock(dma_buf);
+
+ return kunmap(mock->pages[page_num]);
+}
+
+static int mock_dmabuf_mmap(struct dma_buf *dma_buf, struct vm_area_struct *vma)
+{
+ return -ENODEV;
+}
+
+static const struct dma_buf_ops mock_dmabuf_ops = {
+ .map_dma_buf = mock_map_dma_buf,
+ .unmap_dma_buf = mock_unmap_dma_buf,
+ .release = mock_dmabuf_release,
+ .kmap = mock_dmabuf_kmap,
+ .kmap_atomic = mock_dmabuf_kmap_atomic,
+ .kunmap = mock_dmabuf_kunmap,
+ .kunmap_atomic = mock_dmabuf_kunmap_atomic,
+ .mmap = mock_dmabuf_mmap,
+ .vmap = mock_dmabuf_vmap,
+ .vunmap = mock_dmabuf_vunmap,
+};
+
+static struct dma_buf *mock_dmabuf(int npages)
+{
+ struct mock_dmabuf *mock;
+ DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+ struct dma_buf *dmabuf;
+ int i;
+
+ mock = kmalloc(sizeof(*mock) + npages * sizeof(struct page *),
+ GFP_KERNEL);
+ if (!mock)
+ return ERR_PTR(-ENOMEM);
+
+ mock->npages = npages;
+ for (i = 0; i < npages; i++) {
+ mock->pages[i] = alloc_page(GFP_KERNEL);
+ if (!mock->pages[i])
+ goto err;
+ }
+
+ exp_info.ops = &mock_dmabuf_ops;
+ exp_info.size = npages * PAGE_SIZE;
+ exp_info.flags = O_CLOEXEC;
+ exp_info.priv = mock;
+
+ dmabuf = dma_buf_export(&exp_info);
+ if (IS_ERR(dmabuf))
+ goto err;
+
+ return dmabuf;
+
+err:
+ while (i--)
+ put_page(mock->pages[i]);
+ kfree(mock);
+ return ERR_PTR(-ENOMEM);
+}
--- /dev/null
+
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __MOCK_DMABUF_H__
+#define __MOCK_DMABUF_H__
+
+#include <linux/dma-buf.h>
+
+struct mock_dmabuf {
+ int npages;
+ struct page *pages[];
+};
+
+static struct mock_dmabuf *to_mock(struct dma_buf *buf)
+{
+ return buf->priv;
+}
+
+#endif /* !__MOCK_DMABUF_H__ */
--- /dev/null
+/*
+ * Copyright © 2017 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "mock_drm.h"
+
+static inline struct inode fake_inode(struct drm_i915_private *i915)
+{
+ return (struct inode){ .i_rdev = i915->drm.primary->index };
+}
+
+struct drm_file *mock_file(struct drm_i915_private *i915)
+{
+ struct inode inode = fake_inode(i915);
+ struct file filp = {};
+ struct drm_file *file;
+ int err;
+
+ err = drm_open(&inode, &filp);
+ if (unlikely(err))
+ return ERR_PTR(err);
+
+ file = filp.private_data;
+ file->authenticated = true;
+ return file;
+}
+
+void mock_file_free(struct drm_i915_private *i915, struct drm_file *file)
+{
+ struct inode inode = fake_inode(i915);
+ struct file filp = { .private_data = file };
+
+ drm_release(&inode, &filp);
+}
--- /dev/null
+/*
+ * Copyright © 2017 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __MOCK_DRM_H
+#define __MOCK_DRM_H
+
+struct drm_file *mock_file(struct drm_i915_private *i915);
+void mock_file_free(struct drm_i915_private *i915, struct drm_file *file);
+
+#endif /* !__MOCK_DRM_H */
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "mock_engine.h"
+#include "mock_request.h"
+
+static struct mock_request *first_request(struct mock_engine *engine)
+{
+ return list_first_entry_or_null(&engine->hw_queue,
+ struct mock_request,
+ link);
+}
+
+static void hw_delay_complete(unsigned long data)
+{
+ struct mock_engine *engine = (typeof(engine))data;
+ struct mock_request *request;
+
+ spin_lock(&engine->hw_lock);
+
+ request = first_request(engine);
+ if (request) {
+ list_del_init(&request->link);
+ mock_seqno_advance(&engine->base, request->base.global_seqno);
+ }
+
+ request = first_request(engine);
+ if (request)
+ mod_timer(&engine->hw_delay, jiffies + request->delay);
+
+ spin_unlock(&engine->hw_lock);
+}
+
+static int mock_context_pin(struct intel_engine_cs *engine,
+ struct i915_gem_context *ctx)
+{
+ i915_gem_context_get(ctx);
+ return 0;
+}
+
+static void mock_context_unpin(struct intel_engine_cs *engine,
+ struct i915_gem_context *ctx)
+{
+ i915_gem_context_put(ctx);
+}
+
+static int mock_request_alloc(struct drm_i915_gem_request *request)
+{
+ struct mock_request *mock = container_of(request, typeof(*mock), base);
+
+ INIT_LIST_HEAD(&mock->link);
+ mock->delay = 0;
+
+ request->ring = request->engine->buffer;
+ return 0;
+}
+
+static int mock_emit_flush(struct drm_i915_gem_request *request,
+ unsigned int flags)
+{
+ return 0;
+}
+
+static void mock_emit_breadcrumb(struct drm_i915_gem_request *request,
+ u32 *flags)
+{
+}
+
+static void mock_submit_request(struct drm_i915_gem_request *request)
+{
+ struct mock_request *mock = container_of(request, typeof(*mock), base);
+ struct mock_engine *engine =
+ container_of(request->engine, typeof(*engine), base);
+
+ i915_gem_request_submit(request);
+ GEM_BUG_ON(!request->global_seqno);
+
+ spin_lock_irq(&engine->hw_lock);
+ list_add_tail(&mock->link, &engine->hw_queue);
+ if (mock->link.prev == &engine->hw_queue)
+ mod_timer(&engine->hw_delay, jiffies + mock->delay);
+ spin_unlock_irq(&engine->hw_lock);
+}
+
+static struct intel_ring *mock_ring(struct intel_engine_cs *engine)
+{
+ const unsigned long sz = roundup_pow_of_two(sizeof(struct intel_ring));
+ struct intel_ring *ring;
+
+ ring = kzalloc(sizeof(*ring) + sz, GFP_KERNEL);
+ if (!ring)
+ return NULL;
+
+ ring->engine = engine;
+ ring->size = sz;
+ ring->effective_size = sz;
+ ring->vaddr = (void *)(ring + 1);
+
+ INIT_LIST_HEAD(&ring->request_list);
+ ring->last_retired_head = -1;
+ intel_ring_update_space(ring);
+
+ return ring;
+}
+
+struct intel_engine_cs *mock_engine(struct drm_i915_private *i915,
+ const char *name)
+{
+ struct mock_engine *engine;
+ static int id;
+
+ engine = kzalloc(sizeof(*engine) + PAGE_SIZE, GFP_KERNEL);
+ if (!engine)
+ return NULL;
+
+ engine->base.buffer = mock_ring(&engine->base);
+ if (!engine->base.buffer) {
+ kfree(engine);
+ return NULL;
+ }
+
+ /* minimal engine setup for requests */
+ engine->base.i915 = i915;
+ engine->base.name = name;
+ engine->base.id = id++;
+ engine->base.status_page.page_addr = (void *)(engine + 1);
+
+ engine->base.context_pin = mock_context_pin;
+ engine->base.context_unpin = mock_context_unpin;
+ engine->base.request_alloc = mock_request_alloc;
+ engine->base.emit_flush = mock_emit_flush;
+ engine->base.emit_breadcrumb = mock_emit_breadcrumb;
+ engine->base.submit_request = mock_submit_request;
+
+ engine->base.timeline =
+ &i915->gt.global_timeline.engine[engine->base.id];
+
+ intel_engine_init_breadcrumbs(&engine->base);
+ engine->base.breadcrumbs.mock = true; /* prevent touching HW for irqs */
+
+ /* fake hw queue */
+ spin_lock_init(&engine->hw_lock);
+ setup_timer(&engine->hw_delay,
+ hw_delay_complete,
+ (unsigned long)engine);
+ INIT_LIST_HEAD(&engine->hw_queue);
+
+ return &engine->base;
+}
+
+void mock_engine_flush(struct intel_engine_cs *engine)
+{
+ struct mock_engine *mock =
+ container_of(engine, typeof(*mock), base);
+ struct mock_request *request, *rn;
+
+ del_timer_sync(&mock->hw_delay);
+
+ spin_lock_irq(&mock->hw_lock);
+ list_for_each_entry_safe(request, rn, &mock->hw_queue, link) {
+ list_del_init(&request->link);
+ mock_seqno_advance(&mock->base, request->base.global_seqno);
+ }
+ spin_unlock_irq(&mock->hw_lock);
+}
+
+void mock_engine_reset(struct intel_engine_cs *engine)
+{
+ intel_write_status_page(engine, I915_GEM_HWS_INDEX, 0);
+}
+
+void mock_engine_free(struct intel_engine_cs *engine)
+{
+ struct mock_engine *mock =
+ container_of(engine, typeof(*mock), base);
+
+ GEM_BUG_ON(timer_pending(&mock->hw_delay));
+
+ if (engine->last_retired_context)
+ engine->context_unpin(engine, engine->last_retired_context);
+
+ intel_engine_fini_breadcrumbs(engine);
+
+ kfree(engine->buffer);
+ kfree(engine);
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __MOCK_ENGINE_H__
+#define __MOCK_ENGINE_H__
+
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/timer.h>
+
+#include "../intel_ringbuffer.h"
+
+struct mock_engine {
+ struct intel_engine_cs base;
+
+ spinlock_t hw_lock;
+ struct list_head hw_queue;
+ struct timer_list hw_delay;
+};
+
+struct intel_engine_cs *mock_engine(struct drm_i915_private *i915,
+ const char *name);
+void mock_engine_flush(struct intel_engine_cs *engine);
+void mock_engine_reset(struct intel_engine_cs *engine);
+void mock_engine_free(struct intel_engine_cs *engine);
+
+static inline void mock_seqno_advance(struct intel_engine_cs *engine, u32 seqno)
+{
+ intel_write_status_page(engine, I915_GEM_HWS_INDEX, seqno);
+ intel_engine_wakeup(engine);
+}
+
+#endif /* !__MOCK_ENGINE_H__ */
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include <linux/pm_runtime.h>
+
+#include "mock_engine.h"
+#include "mock_context.h"
+#include "mock_request.h"
+#include "mock_gem_device.h"
+#include "mock_gem_object.h"
+#include "mock_gtt.h"
+
+void mock_device_flush(struct drm_i915_private *i915)
+{
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+
+ lockdep_assert_held(&i915->drm.struct_mutex);
+
+ for_each_engine(engine, i915, id)
+ mock_engine_flush(engine);
+
+ i915_gem_retire_requests(i915);
+}
+
+static void mock_device_release(struct drm_device *dev)
+{
+ struct drm_i915_private *i915 = to_i915(dev);
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+
+ mutex_lock(&i915->drm.struct_mutex);
+ mock_device_flush(i915);
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ cancel_delayed_work_sync(&i915->gt.retire_work);
+ cancel_delayed_work_sync(&i915->gt.idle_work);
+
+ mutex_lock(&i915->drm.struct_mutex);
+ for_each_engine(engine, i915, id)
+ mock_engine_free(engine);
+ i915_gem_context_fini(i915);
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ drain_workqueue(i915->wq);
+ i915_gem_drain_freed_objects(i915);
+
+ mutex_lock(&i915->drm.struct_mutex);
+ mock_fini_ggtt(i915);
+ i915_gem_timeline_fini(&i915->gt.global_timeline);
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ destroy_workqueue(i915->wq);
+
+ kmem_cache_destroy(i915->dependencies);
+ kmem_cache_destroy(i915->requests);
+ kmem_cache_destroy(i915->vmas);
+ kmem_cache_destroy(i915->objects);
+
+ drm_dev_fini(&i915->drm);
+ put_device(&i915->drm.pdev->dev);
+}
+
+static struct drm_driver mock_driver = {
+ .name = "mock",
+ .driver_features = DRIVER_GEM,
+ .release = mock_device_release,
+
+ .gem_close_object = i915_gem_close_object,
+ .gem_free_object_unlocked = i915_gem_free_object,
+};
+
+static void release_dev(struct device *dev)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+
+ kfree(pdev);
+}
+
+static void mock_retire_work_handler(struct work_struct *work)
+{
+}
+
+static void mock_idle_work_handler(struct work_struct *work)
+{
+}
+
+struct drm_i915_private *mock_gem_device(void)
+{
+ struct drm_i915_private *i915;
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ struct pci_dev *pdev;
+ int err;
+
+ pdev = kzalloc(sizeof(*pdev) + sizeof(*i915), GFP_KERNEL);
+ if (!pdev)
+ goto err;
+
+ device_initialize(&pdev->dev);
+ pdev->dev.release = release_dev;
+ dev_set_name(&pdev->dev, "mock");
+ dma_coerce_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+
+ pm_runtime_dont_use_autosuspend(&pdev->dev);
+ pm_runtime_get_sync(&pdev->dev);
+
+ i915 = (struct drm_i915_private *)(pdev + 1);
+ pci_set_drvdata(pdev, i915);
+
+ err = drm_dev_init(&i915->drm, &mock_driver, &pdev->dev);
+ if (err) {
+ pr_err("Failed to initialise mock GEM device: err=%d\n", err);
+ goto put_device;
+ }
+ i915->drm.pdev = pdev;
+ i915->drm.dev_private = i915;
+
+ /* Using the global GTT may ask questions about KMS users, so prepare */
+ drm_mode_config_init(&i915->drm);
+
+ mkwrite_device_info(i915)->gen = -1;
+
+ spin_lock_init(&i915->mm.object_stat_lock);
+
+ init_waitqueue_head(&i915->gpu_error.wait_queue);
+ init_waitqueue_head(&i915->gpu_error.reset_queue);
+
+ i915->wq = alloc_ordered_workqueue("mock", 0);
+ if (!i915->wq)
+ goto put_device;
+
+ INIT_WORK(&i915->mm.free_work, __i915_gem_free_work);
+ init_llist_head(&i915->mm.free_list);
+ INIT_LIST_HEAD(&i915->mm.unbound_list);
+ INIT_LIST_HEAD(&i915->mm.bound_list);
+
+ ida_init(&i915->context_hw_ida);
+
+ INIT_DELAYED_WORK(&i915->gt.retire_work, mock_retire_work_handler);
+ INIT_DELAYED_WORK(&i915->gt.idle_work, mock_idle_work_handler);
+
+ i915->gt.awake = true;
+
+ i915->objects = KMEM_CACHE(mock_object, SLAB_HWCACHE_ALIGN);
+ if (!i915->objects)
+ goto err_wq;
+
+ i915->vmas = KMEM_CACHE(i915_vma, SLAB_HWCACHE_ALIGN);
+ if (!i915->vmas)
+ goto err_objects;
+
+ i915->requests = KMEM_CACHE(mock_request,
+ SLAB_HWCACHE_ALIGN |
+ SLAB_RECLAIM_ACCOUNT |
+ SLAB_DESTROY_BY_RCU);
+ if (!i915->requests)
+ goto err_vmas;
+
+ i915->dependencies = KMEM_CACHE(i915_dependency,
+ SLAB_HWCACHE_ALIGN |
+ SLAB_RECLAIM_ACCOUNT);
+ if (!i915->dependencies)
+ goto err_requests;
+
+ mutex_lock(&i915->drm.struct_mutex);
+ INIT_LIST_HEAD(&i915->gt.timelines);
+ err = i915_gem_timeline_init__global(i915);
+ if (err) {
+ mutex_unlock(&i915->drm.struct_mutex);
+ goto err_dependencies;
+ }
+
+ mock_init_ggtt(i915);
+ mutex_unlock(&i915->drm.struct_mutex);
+
+ mkwrite_device_info(i915)->ring_mask = BIT(0);
+ i915->engine[RCS] = mock_engine(i915, "mock");
+ if (!i915->engine[RCS])
+ goto err_dependencies;
+
+ i915->kernel_context = mock_context(i915, NULL);
+ if (!i915->kernel_context)
+ goto err_engine;
+
+ return i915;
+
+err_engine:
+ for_each_engine(engine, i915, id)
+ mock_engine_free(engine);
+err_dependencies:
+ kmem_cache_destroy(i915->dependencies);
+err_requests:
+ kmem_cache_destroy(i915->requests);
+err_vmas:
+ kmem_cache_destroy(i915->vmas);
+err_objects:
+ kmem_cache_destroy(i915->objects);
+err_wq:
+ destroy_workqueue(i915->wq);
+put_device:
+ put_device(&pdev->dev);
+err:
+ return NULL;
+}
--- /dev/null
+#ifndef __MOCK_GEM_DEVICE_H__
+#define __MOCK_GEM_DEVICE_H__
+
+struct drm_i915_private;
+
+struct drm_i915_private *mock_gem_device(void);
+void mock_device_flush(struct drm_i915_private *i915);
+
+#endif /* !__MOCK_GEM_DEVICE_H__ */
--- /dev/null
+#ifndef __MOCK_GEM_OBJECT_H__
+#define __MOCK_GEM_OBJECT_H__
+
+struct mock_object {
+ struct drm_i915_gem_object base;
+};
+
+#endif /* !__MOCK_GEM_OBJECT_H__ */
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "mock_gtt.h"
+
+static void mock_insert_page(struct i915_address_space *vm,
+ dma_addr_t addr,
+ u64 offset,
+ enum i915_cache_level level,
+ u32 flags)
+{
+}
+
+static void mock_insert_entries(struct i915_address_space *vm,
+ struct sg_table *st,
+ u64 start,
+ enum i915_cache_level level, u32 flags)
+{
+}
+
+static int mock_bind_ppgtt(struct i915_vma *vma,
+ enum i915_cache_level cache_level,
+ u32 flags)
+{
+ GEM_BUG_ON(flags & I915_VMA_GLOBAL_BIND);
+ vma->pages = vma->obj->mm.pages;
+ vma->flags |= I915_VMA_LOCAL_BIND;
+ return 0;
+}
+
+static void mock_unbind_ppgtt(struct i915_vma *vma)
+{
+}
+
+static void mock_cleanup(struct i915_address_space *vm)
+{
+}
+
+struct i915_hw_ppgtt *
+mock_ppgtt(struct drm_i915_private *i915,
+ const char *name)
+{
+ struct i915_hw_ppgtt *ppgtt;
+
+ ppgtt = kzalloc(sizeof(*ppgtt), GFP_KERNEL);
+ if (!ppgtt)
+ return NULL;
+
+ kref_init(&ppgtt->ref);
+ ppgtt->base.i915 = i915;
+ ppgtt->base.total = round_down(U64_MAX, PAGE_SIZE);
+ ppgtt->base.file = ERR_PTR(-ENODEV);
+
+ INIT_LIST_HEAD(&ppgtt->base.active_list);
+ INIT_LIST_HEAD(&ppgtt->base.inactive_list);
+ INIT_LIST_HEAD(&ppgtt->base.unbound_list);
+
+ INIT_LIST_HEAD(&ppgtt->base.global_link);
+ drm_mm_init(&ppgtt->base.mm, 0, ppgtt->base.total);
+ i915_gem_timeline_init(i915, &ppgtt->base.timeline, name);
+
+ ppgtt->base.clear_range = nop_clear_range;
+ ppgtt->base.insert_page = mock_insert_page;
+ ppgtt->base.insert_entries = mock_insert_entries;
+ ppgtt->base.bind_vma = mock_bind_ppgtt;
+ ppgtt->base.unbind_vma = mock_unbind_ppgtt;
+ ppgtt->base.cleanup = mock_cleanup;
+
+ return ppgtt;
+}
+
+static int mock_bind_ggtt(struct i915_vma *vma,
+ enum i915_cache_level cache_level,
+ u32 flags)
+{
+ int err;
+
+ err = i915_get_ggtt_vma_pages(vma);
+ if (err)
+ return err;
+
+ vma->flags |= I915_VMA_GLOBAL_BIND | I915_VMA_LOCAL_BIND;
+ return 0;
+}
+
+static void mock_unbind_ggtt(struct i915_vma *vma)
+{
+}
+
+void mock_init_ggtt(struct drm_i915_private *i915)
+{
+ struct i915_ggtt *ggtt = &i915->ggtt;
+
+ INIT_LIST_HEAD(&i915->vm_list);
+
+ ggtt->base.i915 = i915;
+
+ ggtt->mappable_base = 0;
+ ggtt->mappable_end = 2048 * PAGE_SIZE;
+ ggtt->base.total = 4096 * PAGE_SIZE;
+
+ ggtt->base.clear_range = nop_clear_range;
+ ggtt->base.insert_page = mock_insert_page;
+ ggtt->base.insert_entries = mock_insert_entries;
+ ggtt->base.bind_vma = mock_bind_ggtt;
+ ggtt->base.unbind_vma = mock_unbind_ggtt;
+ ggtt->base.cleanup = mock_cleanup;
+
+ i915_address_space_init(&ggtt->base, i915, "global");
+}
+
+void mock_fini_ggtt(struct drm_i915_private *i915)
+{
+ struct i915_ggtt *ggtt = &i915->ggtt;
+
+ i915_address_space_fini(&ggtt->base);
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __MOCK_GTT_H
+#define __MOCK_GTT_H
+
+void mock_init_ggtt(struct drm_i915_private *i915);
+void mock_fini_ggtt(struct drm_i915_private *i915);
+
+struct i915_hw_ppgtt *
+mock_ppgtt(struct drm_i915_private *i915,
+ const char *name);
+
+#endif /* !__MOCK_GTT_H */
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "mock_engine.h"
+#include "mock_request.h"
+
+struct drm_i915_gem_request *
+mock_request(struct intel_engine_cs *engine,
+ struct i915_gem_context *context,
+ unsigned long delay)
+{
+ struct drm_i915_gem_request *request;
+ struct mock_request *mock;
+
+ /* NB the i915->requests slab cache is enlarged to fit mock_request */
+ request = i915_gem_request_alloc(engine, context);
+ if (!request)
+ return NULL;
+
+ mock = container_of(request, typeof(*mock), base);
+ mock->delay = delay;
+
+ return &mock->base;
+}
+
+bool mock_cancel_request(struct drm_i915_gem_request *request)
+{
+ struct mock_request *mock = container_of(request, typeof(*mock), base);
+ struct mock_engine *engine =
+ container_of(request->engine, typeof(*engine), base);
+ bool was_queued;
+
+ spin_lock_irq(&engine->hw_lock);
+ was_queued = !list_empty(&mock->link);
+ list_del_init(&mock->link);
+ spin_unlock_irq(&engine->hw_lock);
+
+ if (was_queued)
+ i915_gem_request_unsubmit(request);
+
+ return was_queued;
+}
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __MOCK_REQUEST__
+#define __MOCK_REQUEST__
+
+#include <linux/list.h>
+
+#include "../i915_gem_request.h"
+
+struct mock_request {
+ struct drm_i915_gem_request base;
+
+ struct list_head link;
+ unsigned long delay;
+};
+
+struct drm_i915_gem_request *
+mock_request(struct intel_engine_cs *engine,
+ struct i915_gem_context *context,
+ unsigned long delay);
+
+bool mock_cancel_request(struct drm_i915_gem_request *request);
+
+#endif /* !__MOCK_REQUEST__ */
--- /dev/null
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#include <linux/prime_numbers.h>
+#include <linux/random.h>
+
+#include "../i915_selftest.h"
+
+#define PFN_BIAS (1 << 10)
+
+struct pfn_table {
+ struct sg_table st;
+ unsigned long start, end;
+};
+
+typedef unsigned int (*npages_fn_t)(unsigned long n,
+ unsigned long count,
+ struct rnd_state *rnd);
+
+static noinline int expect_pfn_sg(struct pfn_table *pt,
+ npages_fn_t npages_fn,
+ struct rnd_state *rnd,
+ const char *who,
+ unsigned long timeout)
+{
+ struct scatterlist *sg;
+ unsigned long pfn, n;
+
+ pfn = pt->start;
+ for_each_sg(pt->st.sgl, sg, pt->st.nents, n) {
+ struct page *page = sg_page(sg);
+ unsigned int npages = npages_fn(n, pt->st.nents, rnd);
+
+ if (page_to_pfn(page) != pfn) {
+ pr_err("%s: %s left pages out of order, expected pfn %lu, found pfn %lu (using for_each_sg)\n",
+ __func__, who, pfn, page_to_pfn(page));
+ return -EINVAL;
+ }
+
+ if (sg->length != npages * PAGE_SIZE) {
+ pr_err("%s: %s copied wrong sg length, expected size %lu, found %u (using for_each_sg)\n",
+ __func__, who, npages * PAGE_SIZE, sg->length);
+ return -EINVAL;
+ }
+
+ if (igt_timeout(timeout, "%s timed out\n", who))
+ return -EINTR;
+
+ pfn += npages;
+ }
+ if (pfn != pt->end) {
+ pr_err("%s: %s finished on wrong pfn, expected %lu, found %lu\n",
+ __func__, who, pt->end, pfn);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static noinline int expect_pfn_sg_page_iter(struct pfn_table *pt,
+ const char *who,
+ unsigned long timeout)
+{
+ struct sg_page_iter sgiter;
+ unsigned long pfn;
+
+ pfn = pt->start;
+ for_each_sg_page(pt->st.sgl, &sgiter, pt->st.nents, 0) {
+ struct page *page = sg_page_iter_page(&sgiter);
+
+ if (page != pfn_to_page(pfn)) {
+ pr_err("%s: %s left pages out of order, expected pfn %lu, found pfn %lu (using for_each_sg_page)\n",
+ __func__, who, pfn, page_to_pfn(page));
+ return -EINVAL;
+ }
+
+ if (igt_timeout(timeout, "%s timed out\n", who))
+ return -EINTR;
+
+ pfn++;
+ }
+ if (pfn != pt->end) {
+ pr_err("%s: %s finished on wrong pfn, expected %lu, found %lu\n",
+ __func__, who, pt->end, pfn);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static noinline int expect_pfn_sgtiter(struct pfn_table *pt,
+ const char *who,
+ unsigned long timeout)
+{
+ struct sgt_iter sgt;
+ struct page *page;
+ unsigned long pfn;
+
+ pfn = pt->start;
+ for_each_sgt_page(page, sgt, &pt->st) {
+ if (page != pfn_to_page(pfn)) {
+ pr_err("%s: %s left pages out of order, expected pfn %lu, found pfn %lu (using for_each_sgt_page)\n",
+ __func__, who, pfn, page_to_pfn(page));
+ return -EINVAL;
+ }
+
+ if (igt_timeout(timeout, "%s timed out\n", who))
+ return -EINTR;
+
+ pfn++;
+ }
+ if (pfn != pt->end) {
+ pr_err("%s: %s finished on wrong pfn, expected %lu, found %lu\n",
+ __func__, who, pt->end, pfn);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int expect_pfn_sgtable(struct pfn_table *pt,
+ npages_fn_t npages_fn,
+ struct rnd_state *rnd,
+ const char *who,
+ unsigned long timeout)
+{
+ int err;
+
+ err = expect_pfn_sg(pt, npages_fn, rnd, who, timeout);
+ if (err)
+ return err;
+
+ err = expect_pfn_sg_page_iter(pt, who, timeout);
+ if (err)
+ return err;
+
+ err = expect_pfn_sgtiter(pt, who, timeout);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static unsigned int one(unsigned long n,
+ unsigned long count,
+ struct rnd_state *rnd)
+{
+ return 1;
+}
+
+static unsigned int grow(unsigned long n,
+ unsigned long count,
+ struct rnd_state *rnd)
+{
+ return n + 1;
+}
+
+static unsigned int shrink(unsigned long n,
+ unsigned long count,
+ struct rnd_state *rnd)
+{
+ return count - n;
+}
+
+static unsigned int random(unsigned long n,
+ unsigned long count,
+ struct rnd_state *rnd)
+{
+ return 1 + (prandom_u32_state(rnd) % 1024);
+}
+
+static int alloc_table(struct pfn_table *pt,
+ unsigned long count, unsigned long max,
+ npages_fn_t npages_fn,
+ struct rnd_state *rnd,
+ int alloc_error)
+{
+ struct scatterlist *sg;
+ unsigned long n, pfn;
+
+ if (sg_alloc_table(&pt->st, max,
+ GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN))
+ return alloc_error;
+
+ /* count should be less than 20 to prevent overflowing sg->length */
+ GEM_BUG_ON(overflows_type(count * PAGE_SIZE, sg->length));
+
+ /* Construct a table where each scatterlist contains different number
+ * of entries. The idea is to check that we can iterate the individual
+ * pages from inside the coalesced lists.
+ */
+ pt->start = PFN_BIAS;
+ pfn = pt->start;
+ sg = pt->st.sgl;
+ for (n = 0; n < count; n++) {
+ unsigned long npages = npages_fn(n, count, rnd);
+
+ /* Nobody expects the Sparse Memmap! */
+ if (pfn_to_page(pfn + npages) != pfn_to_page(pfn) + npages) {
+ sg_free_table(&pt->st);
+ return -ENOSPC;
+ }
+
+ if (n)
+ sg = sg_next(sg);
+ sg_set_page(sg, pfn_to_page(pfn), npages * PAGE_SIZE, 0);
+
+ GEM_BUG_ON(page_to_pfn(sg_page(sg)) != pfn);
+ GEM_BUG_ON(sg->length != npages * PAGE_SIZE);
+ GEM_BUG_ON(sg->offset != 0);
+
+ pfn += npages;
+ }
+ sg_mark_end(sg);
+ pt->st.nents = n;
+ pt->end = pfn;
+
+ return 0;
+}
+
+static const npages_fn_t npages_funcs[] = {
+ one,
+ grow,
+ shrink,
+ random,
+ NULL,
+};
+
+static int igt_sg_alloc(void *ignored)
+{
+ IGT_TIMEOUT(end_time);
+ const unsigned long max_order = 20; /* approximating a 4GiB object */
+ struct rnd_state prng;
+ unsigned long prime;
+ int alloc_error = -ENOMEM;
+
+ for_each_prime_number(prime, max_order) {
+ unsigned long size = BIT(prime);
+ int offset;
+
+ for (offset = -1; offset <= 1; offset++) {
+ unsigned long sz = size + offset;
+ const npages_fn_t *npages;
+ struct pfn_table pt;
+ int err;
+
+ for (npages = npages_funcs; *npages; npages++) {
+ prandom_seed_state(&prng,
+ i915_selftest.random_seed);
+ err = alloc_table(&pt, sz, sz, *npages, &prng,
+ alloc_error);
+ if (err == -ENOSPC)
+ break;
+ if (err)
+ return err;
+
+ prandom_seed_state(&prng,
+ i915_selftest.random_seed);
+ err = expect_pfn_sgtable(&pt, *npages, &prng,
+ "sg_alloc_table",
+ end_time);
+ sg_free_table(&pt.st);
+ if (err)
+ return err;
+ }
+ }
+
+ /* Test at least one continuation before accepting oom */
+ if (size > SG_MAX_SINGLE_ALLOC)
+ alloc_error = -ENOSPC;
+ }
+
+ return 0;
+}
+
+static int igt_sg_trim(void *ignored)
+{
+ IGT_TIMEOUT(end_time);
+ const unsigned long max = PAGE_SIZE; /* not prime! */
+ struct pfn_table pt;
+ unsigned long prime;
+ int alloc_error = -ENOMEM;
+
+ for_each_prime_number(prime, max) {
+ const npages_fn_t *npages;
+ int err;
+
+ for (npages = npages_funcs; *npages; npages++) {
+ struct rnd_state prng;
+
+ prandom_seed_state(&prng, i915_selftest.random_seed);
+ err = alloc_table(&pt, prime, max, *npages, &prng,
+ alloc_error);
+ if (err == -ENOSPC)
+ break;
+ if (err)
+ return err;
+
+ if (i915_sg_trim(&pt.st)) {
+ if (pt.st.orig_nents != prime ||
+ pt.st.nents != prime) {
+ pr_err("i915_sg_trim failed (nents %u, orig_nents %u), expected %lu\n",
+ pt.st.nents, pt.st.orig_nents, prime);
+ err = -EINVAL;
+ } else {
+ prandom_seed_state(&prng,
+ i915_selftest.random_seed);
+ err = expect_pfn_sgtable(&pt,
+ *npages, &prng,
+ "i915_sg_trim",
+ end_time);
+ }
+ }
+ sg_free_table(&pt.st);
+ if (err)
+ return err;
+ }
+
+ /* Test at least one continuation before accepting oom */
+ if (prime > SG_MAX_SINGLE_ALLOC)
+ alloc_error = -ENOSPC;
+ }
+
+ return 0;
+}
+
+int scatterlist_mock_selftests(void)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(igt_sg_alloc),
+ SUBTEST(igt_sg_trim),
+ };
+
+ return i915_subtests(tests, NULL);
+}
.mpll_cfg = imx_mpll_cfg,
.cur_ctr = imx_cur_ctr,
.phy_config = imx_phy_config,
- .dev_type = IMX6Q_HDMI,
.mode_valid = imx6q_hdmi_mode_valid,
};
.mpll_cfg = imx_mpll_cfg,
.cur_ctr = imx_cur_ctr,
.phy_config = imx_phy_config,
- .dev_type = IMX6DL_HDMI,
.mode_valid = imx6dl_hdmi_mode_valid,
};
#include <video/imx-ipu-v3.h>
#include "imx-drm.h"
+#include "ipuv3-plane.h"
#define MAX_CRTC 4
-struct imx_drm_component {
- struct device_node *of_node;
- struct list_head list;
-};
-
struct imx_drm_device {
struct drm_device *drm;
- struct imx_drm_crtc *crtc[MAX_CRTC];
unsigned int pipes;
struct drm_fbdev_cma *fbhelper;
struct drm_atomic_state *state;
};
-struct imx_drm_crtc {
- struct drm_crtc *crtc;
- struct imx_drm_crtc_helper_funcs imx_drm_helper_funcs;
-};
-
#if IS_ENABLED(CONFIG_DRM_FBDEV_EMULATION)
static int legacyfb_depth = 16;
module_param(legacyfb_depth, int, 0444);
drm_fbdev_cma_restore_mode(imxdrm->fbhelper);
}
-static int imx_drm_enable_vblank(struct drm_device *drm, unsigned int pipe)
-{
- struct imx_drm_device *imxdrm = drm->dev_private;
- struct imx_drm_crtc *imx_drm_crtc = imxdrm->crtc[pipe];
- int ret;
-
- if (!imx_drm_crtc)
- return -EINVAL;
-
- if (!imx_drm_crtc->imx_drm_helper_funcs.enable_vblank)
- return -ENOSYS;
-
- ret = imx_drm_crtc->imx_drm_helper_funcs.enable_vblank(
- imx_drm_crtc->crtc);
-
- return ret;
-}
-
-static void imx_drm_disable_vblank(struct drm_device *drm, unsigned int pipe)
-{
- struct imx_drm_device *imxdrm = drm->dev_private;
- struct imx_drm_crtc *imx_drm_crtc = imxdrm->crtc[pipe];
-
- if (!imx_drm_crtc)
- return;
-
- if (!imx_drm_crtc->imx_drm_helper_funcs.disable_vblank)
- return;
-
- imx_drm_crtc->imx_drm_helper_funcs.disable_vblank(imx_drm_crtc->crtc);
-}
-
-static const struct file_operations imx_drm_driver_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
- .mmap = drm_gem_cma_mmap,
- .poll = drm_poll,
- .read = drm_read,
- .llseek = noop_llseek,
-};
+DEFINE_DRM_GEM_CMA_FOPS(imx_drm_driver_fops);
void imx_drm_connector_destroy(struct drm_connector *connector)
{
if (ret)
return ret;
+ /* Assign PRG/PRE channels and check if all constrains are satisfied. */
+ ret = ipu_planes_assign_pre(dev, state);
+ if (ret)
+ return ret;
+
return ret;
}
static void imx_drm_atomic_commit_tail(struct drm_atomic_state *state)
{
struct drm_device *dev = state->dev;
+ struct drm_plane *plane;
+ struct drm_plane_state *old_plane_state;
+ bool plane_disabling = false;
+ int i;
drm_atomic_helper_commit_modeset_disables(dev, state);
drm_atomic_helper_commit_modeset_enables(dev, state);
- drm_atomic_helper_commit_hw_done(state);
-
- drm_atomic_helper_wait_for_vblanks(dev, state);
-
- drm_atomic_helper_cleanup_planes(dev, state);
-}
-
-static struct drm_mode_config_helper_funcs imx_drm_mode_config_helpers = {
- .atomic_commit_tail = imx_drm_atomic_commit_tail,
-};
-
-/*
- * imx_drm_add_crtc - add a new crtc
- */
-int imx_drm_add_crtc(struct drm_device *drm, struct drm_crtc *crtc,
- struct imx_drm_crtc **new_crtc, struct drm_plane *primary_plane,
- const struct imx_drm_crtc_helper_funcs *imx_drm_helper_funcs,
- struct device_node *port)
-{
- struct imx_drm_device *imxdrm = drm->dev_private;
- struct imx_drm_crtc *imx_drm_crtc;
-
- /*
- * The vblank arrays are dimensioned by MAX_CRTC - we can't
- * pass IDs greater than this to those functions.
- */
- if (imxdrm->pipes >= MAX_CRTC)
- return -EINVAL;
-
- if (imxdrm->drm->open_count)
- return -EBUSY;
-
- imx_drm_crtc = kzalloc(sizeof(*imx_drm_crtc), GFP_KERNEL);
- if (!imx_drm_crtc)
- return -ENOMEM;
-
- imx_drm_crtc->imx_drm_helper_funcs = *imx_drm_helper_funcs;
- imx_drm_crtc->crtc = crtc;
-
- crtc->port = port;
-
- imxdrm->crtc[imxdrm->pipes++] = imx_drm_crtc;
+ for_each_plane_in_state(state, plane, old_plane_state, i) {
+ if (drm_atomic_plane_disabling(old_plane_state, plane->state))
+ plane_disabling = true;
+ }
- *new_crtc = imx_drm_crtc;
+ if (plane_disabling) {
+ drm_atomic_helper_wait_for_vblanks(dev, state);
- drm_crtc_helper_add(crtc,
- imx_drm_crtc->imx_drm_helper_funcs.crtc_helper_funcs);
+ for_each_plane_in_state(state, plane, old_plane_state, i)
+ ipu_plane_disable_deferred(plane);
- drm_crtc_init_with_planes(drm, crtc, primary_plane, NULL,
- imx_drm_crtc->imx_drm_helper_funcs.crtc_funcs, NULL);
+ }
- return 0;
+ drm_atomic_helper_commit_hw_done(state);
}
-EXPORT_SYMBOL_GPL(imx_drm_add_crtc);
-
-/*
- * imx_drm_remove_crtc - remove a crtc
- */
-int imx_drm_remove_crtc(struct imx_drm_crtc *imx_drm_crtc)
-{
- struct imx_drm_device *imxdrm = imx_drm_crtc->crtc->dev->dev_private;
- unsigned int pipe = drm_crtc_index(imx_drm_crtc->crtc);
-
- drm_crtc_cleanup(imx_drm_crtc->crtc);
- imxdrm->crtc[pipe] = NULL;
-
- kfree(imx_drm_crtc);
+static const struct drm_mode_config_helper_funcs imx_drm_mode_config_helpers = {
+ .atomic_commit_tail = imx_drm_atomic_commit_tail,
+};
- return 0;
-}
-EXPORT_SYMBOL_GPL(imx_drm_remove_crtc);
int imx_drm_encoder_parse_of(struct drm_device *drm,
struct drm_encoder *encoder, struct device_node *np)
.gem_prime_vmap = drm_gem_cma_prime_vmap,
.gem_prime_vunmap = drm_gem_cma_prime_vunmap,
.gem_prime_mmap = drm_gem_cma_prime_mmap,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = imx_drm_enable_vblank,
- .disable_vblank = imx_drm_disable_vblank,
.ioctls = imx_drm_ioctls,
.num_ioctls = ARRAY_SIZE(imx_drm_ioctls),
.fops = &imx_drm_driver_fops,
{
return container_of(s, struct imx_crtc_state, base);
}
-
-struct imx_drm_crtc_helper_funcs {
- int (*enable_vblank)(struct drm_crtc *crtc);
- void (*disable_vblank)(struct drm_crtc *crtc);
- const struct drm_crtc_helper_funcs *crtc_helper_funcs;
- const struct drm_crtc_funcs *crtc_funcs;
-};
-
-int imx_drm_add_crtc(struct drm_device *drm, struct drm_crtc *crtc,
- struct imx_drm_crtc **new_crtc, struct drm_plane *primary_plane,
- const struct imx_drm_crtc_helper_funcs *imx_helper_funcs,
- struct device_node *port);
-int imx_drm_remove_crtc(struct imx_drm_crtc *);
int imx_drm_init_drm(struct platform_device *pdev,
int preferred_bpp);
int imx_drm_exit_drm(void);
void imx_drm_connector_destroy(struct drm_connector *connector);
void imx_drm_encoder_destroy(struct drm_encoder *encoder);
+int ipu_planes_assign_pre(struct drm_device *dev,
+ struct drm_atomic_state *state);
+
#endif /* _IMX_DRM_H_ */
struct ipu_crtc *ipu_crtc = to_ipu_crtc(crtc);
struct ipu_soc *ipu = dev_get_drvdata(ipu_crtc->dev->parent);
+ ipu_prg_enable(ipu);
ipu_dc_enable(ipu);
ipu_dc_enable_channel(ipu_crtc->dc);
ipu_di_enable(ipu_crtc->di);
}
+static void ipu_crtc_disable_planes(struct ipu_crtc *ipu_crtc,
+ struct drm_crtc_state *old_crtc_state)
+{
+ bool disable_partial = false;
+ bool disable_full = false;
+ struct drm_plane *plane;
+
+ drm_atomic_crtc_state_for_each_plane(plane, old_crtc_state) {
+ if (plane == &ipu_crtc->plane[0]->base)
+ disable_full = true;
+ if (&ipu_crtc->plane[1] && plane == &ipu_crtc->plane[1]->base)
+ disable_partial = true;
+ }
+
+ if (disable_partial)
+ ipu_plane_disable(ipu_crtc->plane[1], true);
+ if (disable_full)
+ ipu_plane_disable(ipu_crtc->plane[0], false);
+}
+
static void ipu_crtc_atomic_disable(struct drm_crtc *crtc,
struct drm_crtc_state *old_crtc_state)
{
* attached IDMACs will be left in undefined state, possibly hanging
* the IPU or even system.
*/
- drm_atomic_helper_disable_planes_on_crtc(old_crtc_state, false);
+ ipu_crtc_disable_planes(ipu_crtc, old_crtc_state);
ipu_dc_disable(ipu);
+ ipu_prg_disable(ipu);
spin_lock_irq(&crtc->dev->event_lock);
if (crtc->state->event) {
kfree(to_imx_crtc_state(state));
}
-static void imx_drm_crtc_destroy(struct drm_crtc *crtc)
+static int ipu_enable_vblank(struct drm_crtc *crtc)
{
- imx_drm_remove_crtc(to_ipu_crtc(crtc)->imx_crtc);
+ struct ipu_crtc *ipu_crtc = to_ipu_crtc(crtc);
+
+ enable_irq(ipu_crtc->irq);
+
+ return 0;
+}
+
+static void ipu_disable_vblank(struct drm_crtc *crtc)
+{
+ struct ipu_crtc *ipu_crtc = to_ipu_crtc(crtc);
+
+ disable_irq_nosync(ipu_crtc->irq);
}
static const struct drm_crtc_funcs ipu_crtc_funcs = {
.set_config = drm_atomic_helper_set_config,
- .destroy = imx_drm_crtc_destroy,
+ .destroy = drm_crtc_cleanup,
.page_flip = drm_atomic_helper_page_flip,
.reset = imx_drm_crtc_reset,
.atomic_duplicate_state = imx_drm_crtc_duplicate_state,
.atomic_destroy_state = imx_drm_crtc_destroy_state,
+ .enable_vblank = ipu_enable_vblank,
+ .disable_vblank = ipu_disable_vblank,
};
static irqreturn_t ipu_irq_handler(int irq, void *dev_id)
.enable = ipu_crtc_enable,
};
-static int ipu_enable_vblank(struct drm_crtc *crtc)
-{
- struct ipu_crtc *ipu_crtc = to_ipu_crtc(crtc);
-
- enable_irq(ipu_crtc->irq);
-
- return 0;
-}
-
-static void ipu_disable_vblank(struct drm_crtc *crtc)
-{
- struct ipu_crtc *ipu_crtc = to_ipu_crtc(crtc);
-
- disable_irq_nosync(ipu_crtc->irq);
-}
-
-static const struct imx_drm_crtc_helper_funcs ipu_crtc_helper_funcs = {
- .enable_vblank = ipu_enable_vblank,
- .disable_vblank = ipu_disable_vblank,
- .crtc_funcs = &ipu_crtc_funcs,
- .crtc_helper_funcs = &ipu_helper_funcs,
-};
-
static void ipu_put_resources(struct ipu_crtc *ipu_crtc)
{
if (!IS_ERR_OR_NULL(ipu_crtc->dc))
struct ipu_client_platformdata *pdata, struct drm_device *drm)
{
struct ipu_soc *ipu = dev_get_drvdata(ipu_crtc->dev->parent);
+ struct drm_crtc *crtc = &ipu_crtc->base;
int dp = -EINVAL;
int ret;
goto err_put_resources;
}
- ret = imx_drm_add_crtc(drm, &ipu_crtc->base, &ipu_crtc->imx_crtc,
- &ipu_crtc->plane[0]->base, &ipu_crtc_helper_funcs,
- pdata->of_node);
- if (ret) {
- dev_err(ipu_crtc->dev, "adding crtc failed with %d.\n", ret);
- goto err_put_resources;
- }
+ crtc->port = pdata->of_node;
+ drm_crtc_helper_add(crtc, &ipu_helper_funcs);
+ drm_crtc_init_with_planes(drm, crtc, &ipu_crtc->plane[0]->base, NULL,
+ &ipu_crtc_funcs, NULL);
ret = ipu_plane_get_resources(ipu_crtc->plane[0]);
if (ret) {
dev_err(ipu_crtc->dev, "getting plane 0 resources failed with %d.\n",
ret);
- goto err_remove_crtc;
+ goto err_put_resources;
}
/* If this crtc is using the DP, add an overlay plane */
ipu_plane_put_resources(ipu_crtc->plane[1]);
err_put_plane0_res:
ipu_plane_put_resources(ipu_crtc->plane[0]);
-err_remove_crtc:
- imx_drm_remove_crtc(ipu_crtc->imx_crtc);
err_put_resources:
ipu_put_resources(ipu_crtc);
#include "video/imx-ipu-v3.h"
#include "ipuv3-plane.h"
+struct ipu_plane_state {
+ struct drm_plane_state base;
+ bool use_pre;
+};
+
+static inline struct ipu_plane_state *
+to_ipu_plane_state(struct drm_plane_state *p)
+{
+ return container_of(p, struct ipu_plane_state, base);
+}
+
static inline struct ipu_plane *to_ipu_plane(struct drm_plane *p)
{
return container_of(p, struct ipu_plane, base);
DRM_FORMAT_NV12,
DRM_FORMAT_NV16,
DRM_FORMAT_RGB565,
+ DRM_FORMAT_RGB565_A8,
+ DRM_FORMAT_BGR565_A8,
+ DRM_FORMAT_RGB888_A8,
+ DRM_FORMAT_BGR888_A8,
+ DRM_FORMAT_RGBX8888_A8,
+ DRM_FORMAT_BGRX8888_A8,
};
int ipu_plane_irq(struct ipu_plane *ipu_plane)
}
static inline unsigned long
-drm_plane_state_to_eba(struct drm_plane_state *state)
+drm_plane_state_to_eba(struct drm_plane_state *state, int plane)
{
struct drm_framebuffer *fb = state->fb;
struct drm_gem_cma_object *cma_obj;
- int x = state->src_x >> 16;
- int y = state->src_y >> 16;
+ int x = state->src.x1 >> 16;
+ int y = state->src.y1 >> 16;
- cma_obj = drm_fb_cma_get_gem_obj(fb, 0);
+ cma_obj = drm_fb_cma_get_gem_obj(fb, plane);
BUG_ON(!cma_obj);
- return cma_obj->paddr + fb->offsets[0] + fb->pitches[0] * y +
- fb->format->cpp[0] * x;
+ return cma_obj->paddr + fb->offsets[plane] + fb->pitches[plane] * y +
+ fb->format->cpp[plane] * x;
}
static inline unsigned long
{
struct drm_framebuffer *fb = state->fb;
struct drm_gem_cma_object *cma_obj;
- unsigned long eba = drm_plane_state_to_eba(state);
- int x = state->src_x >> 16;
- int y = state->src_y >> 16;
+ unsigned long eba = drm_plane_state_to_eba(state, 0);
+ int x = state->src.x1 >> 16;
+ int y = state->src.y1 >> 16;
cma_obj = drm_fb_cma_get_gem_obj(fb, 1);
BUG_ON(!cma_obj);
{
struct drm_framebuffer *fb = state->fb;
struct drm_gem_cma_object *cma_obj;
- unsigned long eba = drm_plane_state_to_eba(state);
- int x = state->src_x >> 16;
- int y = state->src_y >> 16;
+ unsigned long eba = drm_plane_state_to_eba(state, 0);
+ int x = state->src.x1 >> 16;
+ int y = state->src.y1 >> 16;
cma_obj = drm_fb_cma_get_gem_obj(fb, 2);
BUG_ON(!cma_obj);
ipu_dmfc_put(ipu_plane->dmfc);
if (!IS_ERR_OR_NULL(ipu_plane->ipu_ch))
ipu_idmac_put(ipu_plane->ipu_ch);
+ if (!IS_ERR_OR_NULL(ipu_plane->alpha_ch))
+ ipu_idmac_put(ipu_plane->alpha_ch);
}
int ipu_plane_get_resources(struct ipu_plane *ipu_plane)
{
int ret;
+ int alpha_ch;
ipu_plane->ipu_ch = ipu_idmac_get(ipu_plane->ipu, ipu_plane->dma);
if (IS_ERR(ipu_plane->ipu_ch)) {
return ret;
}
+ alpha_ch = ipu_channel_alpha_channel(ipu_plane->dma);
+ if (alpha_ch >= 0) {
+ ipu_plane->alpha_ch = ipu_idmac_get(ipu_plane->ipu, alpha_ch);
+ if (IS_ERR(ipu_plane->alpha_ch)) {
+ ret = PTR_ERR(ipu_plane->alpha_ch);
+ DRM_ERROR("failed to get alpha idmac channel %d: %d\n",
+ alpha_ch, ret);
+ return ret;
+ }
+ }
+
ipu_plane->dmfc = ipu_dmfc_get(ipu_plane->ipu, ipu_plane->dma);
if (IS_ERR(ipu_plane->dmfc)) {
ret = PTR_ERR(ipu_plane->dmfc);
return ret;
}
+static bool ipu_plane_separate_alpha(struct ipu_plane *ipu_plane)
+{
+ switch (ipu_plane->base.state->fb->format->format) {
+ case DRM_FORMAT_RGB565_A8:
+ case DRM_FORMAT_BGR565_A8:
+ case DRM_FORMAT_RGB888_A8:
+ case DRM_FORMAT_BGR888_A8:
+ case DRM_FORMAT_RGBX8888_A8:
+ case DRM_FORMAT_BGRX8888_A8:
+ return true;
+ default:
+ return false;
+ }
+}
+
static void ipu_plane_enable(struct ipu_plane *ipu_plane)
{
if (ipu_plane->dp)
ipu_dp_enable(ipu_plane->ipu);
ipu_dmfc_enable_channel(ipu_plane->dmfc);
ipu_idmac_enable_channel(ipu_plane->ipu_ch);
+ if (ipu_plane_separate_alpha(ipu_plane))
+ ipu_idmac_enable_channel(ipu_plane->alpha_ch);
if (ipu_plane->dp)
ipu_dp_enable_channel(ipu_plane->dp);
}
-static int ipu_disable_plane(struct drm_plane *plane)
+void ipu_plane_disable(struct ipu_plane *ipu_plane, bool disable_dp_channel)
{
- struct ipu_plane *ipu_plane = to_ipu_plane(plane);
-
DRM_DEBUG_KMS("[%d] %s\n", __LINE__, __func__);
ipu_idmac_wait_busy(ipu_plane->ipu_ch, 50);
- if (ipu_plane->dp)
- ipu_dp_disable_channel(ipu_plane->dp);
+ if (ipu_plane->dp && disable_dp_channel)
+ ipu_dp_disable_channel(ipu_plane->dp, false);
ipu_idmac_disable_channel(ipu_plane->ipu_ch);
+ if (ipu_plane->alpha_ch)
+ ipu_idmac_disable_channel(ipu_plane->alpha_ch);
ipu_dmfc_disable_channel(ipu_plane->dmfc);
if (ipu_plane->dp)
ipu_dp_disable(ipu_plane->ipu);
+ if (ipu_prg_present(ipu_plane->ipu))
+ ipu_prg_channel_disable(ipu_plane->ipu_ch);
+}
- return 0;
+void ipu_plane_disable_deferred(struct drm_plane *plane)
+{
+ struct ipu_plane *ipu_plane = to_ipu_plane(plane);
+
+ if (ipu_plane->disabling) {
+ ipu_plane->disabling = false;
+ ipu_plane_disable(ipu_plane, false);
+ }
}
+EXPORT_SYMBOL_GPL(ipu_plane_disable_deferred);
static void ipu_plane_destroy(struct drm_plane *plane)
{
kfree(ipu_plane);
}
+void ipu_plane_state_reset(struct drm_plane *plane)
+{
+ struct ipu_plane_state *ipu_state;
+
+ if (plane->state) {
+ ipu_state = to_ipu_plane_state(plane->state);
+ __drm_atomic_helper_plane_destroy_state(plane->state);
+ kfree(ipu_state);
+ }
+
+ ipu_state = kzalloc(sizeof(*ipu_state), GFP_KERNEL);
+
+ if (ipu_state) {
+ ipu_state->base.plane = plane;
+ ipu_state->base.rotation = DRM_ROTATE_0;
+ }
+
+ plane->state = &ipu_state->base;
+}
+
+struct drm_plane_state *ipu_plane_duplicate_state(struct drm_plane *plane)
+{
+ struct ipu_plane_state *state;
+
+ if (WARN_ON(!plane->state))
+ return NULL;
+
+ state = kmalloc(sizeof(*state), GFP_KERNEL);
+ if (state)
+ __drm_atomic_helper_plane_duplicate_state(plane, &state->base);
+
+ return &state->base;
+}
+
+void ipu_plane_destroy_state(struct drm_plane *plane,
+ struct drm_plane_state *state)
+{
+ struct ipu_plane_state *ipu_state = to_ipu_plane_state(state);
+
+ __drm_atomic_helper_plane_destroy_state(state);
+ kfree(ipu_state);
+}
+
static const struct drm_plane_funcs ipu_plane_funcs = {
.update_plane = drm_atomic_helper_update_plane,
.disable_plane = drm_atomic_helper_disable_plane,
.destroy = ipu_plane_destroy,
- .reset = drm_atomic_helper_plane_reset,
- .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
+ .reset = ipu_plane_state_reset,
+ .atomic_duplicate_state = ipu_plane_duplicate_state,
+ .atomic_destroy_state = ipu_plane_destroy_state,
};
static int ipu_plane_atomic_check(struct drm_plane *plane,
struct device *dev = plane->dev->dev;
struct drm_framebuffer *fb = state->fb;
struct drm_framebuffer *old_fb = old_state->fb;
- unsigned long eba, ubo, vbo, old_ubo, old_vbo;
+ unsigned long eba, ubo, vbo, old_ubo, old_vbo, alpha_eba;
+ bool can_position = (plane->type == DRM_PLANE_TYPE_OVERLAY);
+ struct drm_rect clip;
int hsub, vsub;
+ int ret;
/* Ok to disable */
if (!fb)
if (WARN_ON(!crtc_state))
return -EINVAL;
+ clip.x1 = 0;
+ clip.y1 = 0;
+ clip.x2 = crtc_state->adjusted_mode.hdisplay;
+ clip.y2 = crtc_state->adjusted_mode.vdisplay;
+ ret = drm_plane_helper_check_state(state, &clip,
+ DRM_PLANE_HELPER_NO_SCALING,
+ DRM_PLANE_HELPER_NO_SCALING,
+ can_position, true);
+ if (ret)
+ return ret;
+
/* CRTC should be enabled */
if (!crtc_state->enable)
return -EINVAL;
- /* no scaling */
- if (state->src_w >> 16 != state->crtc_w ||
- state->src_h >> 16 != state->crtc_h)
- return -EINVAL;
-
switch (plane->type) {
case DRM_PLANE_TYPE_PRIMARY:
- /* full plane doesn't support partial off screen */
- if (state->crtc_x || state->crtc_y ||
- state->crtc_w != crtc_state->adjusted_mode.hdisplay ||
- state->crtc_h != crtc_state->adjusted_mode.vdisplay)
- return -EINVAL;
-
/* full plane minimum width is 13 pixels */
- if (state->crtc_w < 13)
+ if (drm_rect_width(&state->dst) < 13)
return -EINVAL;
break;
case DRM_PLANE_TYPE_OVERLAY:
- if (state->crtc_x < 0 || state->crtc_y < 0)
- return -EINVAL;
-
- if (state->crtc_x + state->crtc_w >
- crtc_state->adjusted_mode.hdisplay)
- return -EINVAL;
- if (state->crtc_y + state->crtc_h >
- crtc_state->adjusted_mode.vdisplay)
- return -EINVAL;
break;
default:
- dev_warn(dev, "Unsupported plane type\n");
+ dev_warn(dev, "Unsupported plane type %d\n", plane->type);
return -EINVAL;
}
- if (state->crtc_h < 2)
+ if (drm_rect_height(&state->dst) < 2)
return -EINVAL;
/*
* callback. The planes will be reenabled in plane's ->atomic_update
* callback.
*/
- if (old_fb && (state->src_w != old_state->src_w ||
- state->src_h != old_state->src_h ||
- fb->format != old_fb->format))
+ if (old_fb &&
+ (drm_rect_width(&state->dst) != drm_rect_width(&old_state->dst) ||
+ drm_rect_height(&state->dst) != drm_rect_height(&old_state->dst) ||
+ fb->format != old_fb->format))
crtc_state->mode_changed = true;
- eba = drm_plane_state_to_eba(state);
+ eba = drm_plane_state_to_eba(state, 0);
if (eba & 0x7)
return -EINVAL;
*/
hsub = drm_format_horz_chroma_subsampling(fb->format->format);
vsub = drm_format_vert_chroma_subsampling(fb->format->format);
- if (((state->src_x >> 16) & (hsub - 1)) ||
- ((state->src_y >> 16) & (vsub - 1)))
+ if (((state->src.x1 >> 16) & (hsub - 1)) ||
+ ((state->src.y1 >> 16) & (vsub - 1)))
return -EINVAL;
+ break;
+ case DRM_FORMAT_RGB565_A8:
+ case DRM_FORMAT_BGR565_A8:
+ case DRM_FORMAT_RGB888_A8:
+ case DRM_FORMAT_BGR888_A8:
+ case DRM_FORMAT_RGBX8888_A8:
+ case DRM_FORMAT_BGRX8888_A8:
+ alpha_eba = drm_plane_state_to_eba(state, 1);
+ if (alpha_eba & 0x7)
+ return -EINVAL;
+
+ if (fb->pitches[1] < 1 || fb->pitches[1] > 16384)
+ return -EINVAL;
+
+ if (old_fb && old_fb->pitches[1] != fb->pitches[1])
+ crtc_state->mode_changed = true;
+ break;
}
return 0;
static void ipu_plane_atomic_disable(struct drm_plane *plane,
struct drm_plane_state *old_state)
{
- ipu_disable_plane(plane);
+ struct ipu_plane *ipu_plane = to_ipu_plane(plane);
+
+ if (ipu_plane->dp)
+ ipu_dp_disable_channel(ipu_plane->dp, true);
+ ipu_plane->disabling = true;
+}
+
+static int ipu_chan_assign_axi_id(int ipu_chan)
+{
+ switch (ipu_chan) {
+ case IPUV3_CHANNEL_MEM_BG_SYNC:
+ return 1;
+ case IPUV3_CHANNEL_MEM_FG_SYNC:
+ return 2;
+ case IPUV3_CHANNEL_MEM_DC_SYNC:
+ return 3;
+ default:
+ return 0;
+ }
}
static void ipu_plane_atomic_update(struct drm_plane *plane,
{
struct ipu_plane *ipu_plane = to_ipu_plane(plane);
struct drm_plane_state *state = plane->state;
+ struct ipu_plane_state *ipu_state = to_ipu_plane_state(state);
struct drm_crtc_state *crtc_state = state->crtc->state;
struct drm_framebuffer *fb = state->fb;
+ struct drm_rect *dst = &state->dst;
unsigned long eba, ubo, vbo;
+ unsigned long alpha_eba = 0;
enum ipu_color_space ics;
+ unsigned int axi_id = 0;
int active;
- eba = drm_plane_state_to_eba(state);
+ if (ipu_plane->dp_flow == IPU_DP_FLOW_SYNC_FG)
+ ipu_dp_set_window_pos(ipu_plane->dp, dst->x1, dst->y1);
+
+ eba = drm_plane_state_to_eba(state, 0);
+
+ /*
+ * Configure PRG channel and attached PRE, this changes the EBA to an
+ * internal SRAM location.
+ */
+ if (ipu_state->use_pre) {
+ axi_id = ipu_chan_assign_axi_id(ipu_plane->dma);
+ ipu_prg_channel_configure(ipu_plane->ipu_ch, axi_id,
+ drm_rect_width(&state->src) >> 16,
+ drm_rect_height(&state->src) >> 16,
+ state->fb->pitches[0],
+ state->fb->format->format, &eba);
+ }
if (old_state->fb && !drm_atomic_crtc_needs_modeset(crtc_state)) {
+ /* nothing to do if PRE is used */
+ if (ipu_state->use_pre)
+ return;
active = ipu_idmac_get_current_buffer(ipu_plane->ipu_ch);
ipu_cpmem_set_buffer(ipu_plane->ipu_ch, !active, eba);
ipu_idmac_select_buffer(ipu_plane->ipu_ch, !active);
+ if (ipu_plane_separate_alpha(ipu_plane)) {
+ active = ipu_idmac_get_current_buffer(ipu_plane->alpha_ch);
+ ipu_cpmem_set_buffer(ipu_plane->alpha_ch, !active,
+ alpha_eba);
+ ipu_idmac_select_buffer(ipu_plane->alpha_ch, !active);
+ }
return;
}
ics = ipu_drm_fourcc_to_colorspace(state->fb->format->format);
ipu_dp_setup_channel(ipu_plane->dp, ics,
IPUV3_COLORSPACE_UNKNOWN);
- ipu_dp_set_window_pos(ipu_plane->dp, state->crtc_x,
- state->crtc_y);
/* Enable local alpha on partial plane */
switch (state->fb->format->format) {
case DRM_FORMAT_ARGB1555:
case DRM_FORMAT_ABGR8888:
case DRM_FORMAT_RGBA8888:
case DRM_FORMAT_BGRA8888:
+ case DRM_FORMAT_RGB565_A8:
+ case DRM_FORMAT_BGR565_A8:
+ case DRM_FORMAT_RGB888_A8:
+ case DRM_FORMAT_BGR888_A8:
+ case DRM_FORMAT_RGBX8888_A8:
+ case DRM_FORMAT_BGRX8888_A8:
ipu_dp_set_global_alpha(ipu_plane->dp, false, 0, false);
break;
default:
}
}
- ipu_dmfc_config_wait4eot(ipu_plane->dmfc, state->crtc_w);
+ ipu_dmfc_config_wait4eot(ipu_plane->dmfc, drm_rect_width(dst));
ipu_cpmem_zero(ipu_plane->ipu_ch);
- ipu_cpmem_set_resolution(ipu_plane->ipu_ch, state->src_w >> 16,
- state->src_h >> 16);
+ ipu_cpmem_set_resolution(ipu_plane->ipu_ch,
+ drm_rect_width(&state->src) >> 16,
+ drm_rect_height(&state->src) >> 16);
ipu_cpmem_set_fmt(ipu_plane->ipu_ch, state->fb->format->format);
ipu_cpmem_set_high_priority(ipu_plane->ipu_ch);
ipu_idmac_set_double_buffer(ipu_plane->ipu_ch, 1);
ipu_cpmem_set_stride(ipu_plane->ipu_ch, state->fb->pitches[0]);
+ ipu_cpmem_set_axi_id(ipu_plane->ipu_ch, axi_id);
switch (fb->format->format) {
case DRM_FORMAT_YUV420:
case DRM_FORMAT_YVU420:
dev_dbg(ipu_plane->base.dev->dev,
"phy = %lu %lu %lu, x = %d, y = %d", eba, ubo, vbo,
- state->src_x >> 16, state->src_y >> 16);
+ state->src.x1 >> 16, state->src.y1 >> 16);
break;
case DRM_FORMAT_NV12:
case DRM_FORMAT_NV16:
dev_dbg(ipu_plane->base.dev->dev,
"phy = %lu %lu, x = %d, y = %d", eba, ubo,
- state->src_x >> 16, state->src_y >> 16);
+ state->src.x1 >> 16, state->src.y1 >> 16);
+ break;
+ case DRM_FORMAT_RGB565_A8:
+ case DRM_FORMAT_BGR565_A8:
+ case DRM_FORMAT_RGB888_A8:
+ case DRM_FORMAT_BGR888_A8:
+ case DRM_FORMAT_RGBX8888_A8:
+ case DRM_FORMAT_BGRX8888_A8:
+ alpha_eba = drm_plane_state_to_eba(state, 1);
+
+ dev_dbg(ipu_plane->base.dev->dev, "phys = %lu %lu, x = %d, y = %d",
+ eba, alpha_eba, state->src.x1 >> 16, state->src.y1 >> 16);
+
+ ipu_cpmem_set_burstsize(ipu_plane->ipu_ch, 16);
+
+ ipu_cpmem_zero(ipu_plane->alpha_ch);
+ ipu_cpmem_set_resolution(ipu_plane->alpha_ch,
+ drm_rect_width(&state->src) >> 16,
+ drm_rect_height(&state->src) >> 16);
+ ipu_cpmem_set_format_passthrough(ipu_plane->alpha_ch, 8);
+ ipu_cpmem_set_high_priority(ipu_plane->alpha_ch);
+ ipu_idmac_set_double_buffer(ipu_plane->alpha_ch, 1);
+ ipu_cpmem_set_stride(ipu_plane->alpha_ch,
+ state->fb->pitches[1]);
+ ipu_cpmem_set_burstsize(ipu_plane->alpha_ch, 16);
+ ipu_cpmem_set_buffer(ipu_plane->alpha_ch, 0, alpha_eba);
+ ipu_cpmem_set_buffer(ipu_plane->alpha_ch, 1, alpha_eba);
break;
default:
dev_dbg(ipu_plane->base.dev->dev, "phys = %lu, x = %d, y = %d",
- eba, state->src_x >> 16, state->src_y >> 16);
+ eba, state->src.x1 >> 16, state->src.y1 >> 16);
break;
}
ipu_cpmem_set_buffer(ipu_plane->ipu_ch, 0, eba);
.atomic_update = ipu_plane_atomic_update,
};
+int ipu_planes_assign_pre(struct drm_device *dev,
+ struct drm_atomic_state *state)
+{
+ struct drm_plane_state *plane_state;
+ struct drm_plane *plane;
+ int available_pres = ipu_prg_max_active_channels();
+ int i;
+
+ for_each_plane_in_state(state, plane, plane_state, i) {
+ struct ipu_plane_state *ipu_state =
+ to_ipu_plane_state(plane_state);
+ struct ipu_plane *ipu_plane = to_ipu_plane(plane);
+
+ if (ipu_prg_present(ipu_plane->ipu) && available_pres &&
+ plane_state->fb &&
+ ipu_prg_format_supported(ipu_plane->ipu,
+ plane_state->fb->format->format,
+ plane_state->fb->modifier)) {
+ ipu_state->use_pre = true;
+ available_pres--;
+ } else {
+ ipu_state->use_pre = false;
+ }
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_planes_assign_pre);
+
struct ipu_plane *ipu_plane_init(struct drm_device *dev, struct ipu_soc *ipu,
int dma, int dp, unsigned int possible_crtcs,
enum drm_plane_type type)
struct ipu_soc *ipu;
struct ipuv3_channel *ipu_ch;
+ struct ipuv3_channel *alpha_ch;
struct dmfc_channel *dmfc;
struct ipu_dp *dp;
int dma;
int dp_flow;
+
+ bool disabling;
};
struct ipu_plane *ipu_plane_init(struct drm_device *dev, struct ipu_soc *ipu,
int ipu_plane_irq(struct ipu_plane *plane);
+void ipu_plane_disable(struct ipu_plane *ipu_plane, bool disable_dp_channel);
+void ipu_plane_disable_deferred(struct drm_plane *plane);
+
#endif
state->pending_config = true;
}
-int mtk_drm_crtc_enable_vblank(struct drm_device *drm, unsigned int pipe)
+static int mtk_drm_crtc_enable_vblank(struct drm_crtc *crtc)
{
- struct drm_crtc *crtc = drm_crtc_from_index(drm, pipe);
struct mtk_drm_crtc *mtk_crtc = to_mtk_crtc(crtc);
struct mtk_ddp_comp *ovl = mtk_crtc->ddp_comp[0];
return 0;
}
-void mtk_drm_crtc_disable_vblank(struct drm_device *drm, unsigned int pipe)
+static void mtk_drm_crtc_disable_vblank(struct drm_crtc *crtc)
{
- struct drm_crtc *crtc = drm_crtc_from_index(drm, pipe);
struct mtk_drm_crtc *mtk_crtc = to_mtk_crtc(crtc);
struct mtk_ddp_comp *ovl = mtk_crtc->ddp_comp[0];
.atomic_duplicate_state = mtk_drm_crtc_duplicate_state,
.atomic_destroy_state = mtk_drm_crtc_destroy_state,
.gamma_set = drm_atomic_helper_legacy_gamma_set,
+ .enable_vblank = mtk_drm_crtc_enable_vblank,
+ .disable_vblank = mtk_drm_crtc_disable_vblank,
};
static const struct drm_crtc_helper_funcs mtk_crtc_helper_funcs = {
#define MTK_MAX_BPC 10
#define MTK_MIN_BPC 3
-int mtk_drm_crtc_enable_vblank(struct drm_device *drm, unsigned int pipe);
-void mtk_drm_crtc_disable_vblank(struct drm_device *drm, unsigned int pipe);
void mtk_drm_crtc_commit(struct drm_crtc *crtc);
void mtk_crtc_ddp_irq(struct drm_crtc *crtc, struct mtk_ddp_comp *ovl);
int mtk_drm_crtc_create(struct drm_device *drm_dev,
.driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_PRIME |
DRIVER_ATOMIC,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = mtk_drm_crtc_enable_vblank,
- .disable_vblank = mtk_drm_crtc_disable_vblank,
-
.gem_free_object_unlocked = mtk_drm_gem_free_object,
.gem_vm_ops = &drm_gem_cma_vm_ops,
.dumb_create = mtk_drm_gem_dumb_create,
#include "meson_crtc.h"
#include "meson_plane.h"
+#include "meson_venc.h"
#include "meson_vpp.h"
#include "meson_viu.h"
#include "meson_registers.h"
/* CRTC */
+static int meson_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ struct meson_crtc *meson_crtc = to_meson_crtc(crtc);
+ struct meson_drm *priv = meson_crtc->priv;
+
+ meson_venc_enable_vsync(priv);
+
+ return 0;
+}
+
+static void meson_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+ struct meson_crtc *meson_crtc = to_meson_crtc(crtc);
+ struct meson_drm *priv = meson_crtc->priv;
+
+ meson_venc_disable_vsync(priv);
+}
+
static const struct drm_crtc_funcs meson_crtc_funcs = {
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
.atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
.page_flip = drm_atomic_helper_page_flip,
.reset = drm_atomic_helper_crtc_reset,
.set_config = drm_atomic_helper_set_config,
+ .enable_vblank = meson_crtc_enable_vblank,
+ .disable_vblank = meson_crtc_disable_vblank,
+
};
static void meson_crtc_enable(struct drm_crtc *crtc)
.fb_create = drm_fb_cma_create,
};
-static int meson_enable_vblank(struct drm_device *dev, unsigned int crtc)
-{
- struct meson_drm *priv = dev->dev_private;
-
- meson_venc_enable_vsync(priv);
-
- return 0;
-}
-
-static void meson_disable_vblank(struct drm_device *dev, unsigned int crtc)
-{
- struct meson_drm *priv = dev->dev_private;
-
- meson_venc_disable_vsync(priv);
-}
-
static irqreturn_t meson_irq(int irq, void *arg)
{
struct drm_device *dev = arg;
return IRQ_HANDLED;
}
-static const struct file_operations fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
-#ifdef CONFIG_COMPAT
- .compat_ioctl = drm_compat_ioctl,
-#endif
- .poll = drm_poll,
- .read = drm_read,
- .llseek = no_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(fops);
static struct drm_driver meson_driver = {
.driver_features = DRIVER_HAVE_IRQ | DRIVER_GEM |
DRIVER_MODESET | DRIVER_PRIME |
DRIVER_ATOMIC,
- /* Vblank */
- .enable_vblank = meson_enable_vblank,
- .disable_vblank = meson_disable_vblank,
- .get_vblank_counter = drm_vblank_no_hw_counter,
-
/* IRQ */
.irq_handler = meson_irq,
ret = mgag200_framebuffer_init(dev, &mfbdev->mfb, &mode_cmd, gobj);
if (ret)
- goto err_framebuffer_init;
+ goto err_alloc_fbi;
mfbdev->sysram = sysram;
mfbdev->size = size;
return 0;
-err_framebuffer_init:
- drm_fb_helper_release_fbi(helper);
err_alloc_fbi:
vfree(sysram);
err_sysram:
struct mga_framebuffer *mfb = &mfbdev->mfb;
drm_fb_helper_unregister_fbi(&mfbdev->helper);
- drm_fb_helper_release_fbi(&mfbdev->helper);
if (mfb->obj) {
drm_gem_object_unreference_unlocked(mfb->obj);
}
if (delta > permitteddelta) {
- printk(KERN_WARNING "PLL delta too large\n");
+ pr_warn("PLL delta too large\n");
return 1;
}
msm_host->rx_buf = devm_kzalloc(&pdev->dev, SZ_4K, GFP_KERNEL);
if (!msm_host->rx_buf) {
+ ret = -ENOMEM;
pr_err("%s: alloc rx temp buf failed\n", __func__);
goto fail;
}
struct msm_kms *mdp4_kms_init(struct drm_device *dev)
{
- struct platform_device *pdev = dev->platformdev;
+ struct platform_device *pdev = to_platform_device(dev->dev);
struct mdp4_platform_config *config = mdp4_get_config(pdev);
struct mdp4_kms *mdp4_kms;
struct msm_kms *kms = NULL;
uint32_t major, uint32_t minor)
{
struct drm_device *dev = mdp5_kms->dev;
- struct platform_device *pdev = dev->platformdev;
+ struct platform_device *pdev = to_platform_device(dev->dev);
struct mdp5_cfg_handler *cfg_handler;
struct mdp5_cfg_platform *pconfig;
int i, ret = 0;
return 0;
}
-
-static void mdp5_kms_debugfs_cleanup(struct msm_kms *kms, struct drm_minor *minor)
-{
- drm_debugfs_remove_files(mdp5_debugfs_list,
- ARRAY_SIZE(mdp5_debugfs_list), minor);
-}
#endif
static const struct mdp_kms_funcs kms_funcs = {
.destroy = mdp5_kms_destroy,
#ifdef CONFIG_DEBUG_FS
.debugfs_init = mdp5_kms_debugfs_init,
- .debugfs_cleanup = mdp5_kms_debugfs_cleanup,
#endif
},
.set_irqmask = mdp5_set_irqmask,
int msm_mdss_init(struct drm_device *dev)
{
- struct platform_device *pdev = dev->platformdev;
+ struct platform_device *pdev = to_platform_device(dev->dev);
struct msm_drm_private *priv = dev->dev_private;
struct msm_mdss *mdss;
int ret;
return ret;
}
-
-void msm_debugfs_cleanup(struct drm_minor *minor)
-{
- struct drm_device *dev = minor->dev;
- struct msm_drm_private *priv = dev->dev_private;
-
- drm_debugfs_remove_files(msm_debugfs_list,
- ARRAY_SIZE(msm_debugfs_list), minor);
- if (!priv)
- return;
-
- if (priv->kms->funcs->debugfs_cleanup)
- priv->kms->funcs->debugfs_cleanup(priv->kms, minor);
-
- msm_rd_debugfs_cleanup(minor);
- msm_perf_debugfs_cleanup(minor);
-}
#endif
#ifdef CONFIG_DEBUG_FS
int msm_debugfs_init(struct drm_minor *minor);
-void msm_debugfs_cleanup(struct drm_minor *minor);
#endif
#endif /* __MSM_DEBUGFS_H__ */
{
u32 val = readl(addr);
if (reglog)
- printk(KERN_ERR "IO:R %p %08x\n", addr, val);
+ pr_err("IO:R %p %08x\n", addr, val);
return val;
}
drm_dev_unregister(ddev);
+ msm_perf_debugfs_cleanup(priv);
+ msm_rd_debugfs_cleanup(priv);
+
#ifdef CONFIG_DRM_FBDEV_EMULATION
if (fbdev && priv->fbdev)
msm_fbdev_free(ddev);
}
platform_set_drvdata(pdev, ddev);
- ddev->platformdev = pdev;
priv = kzalloc(sizeof(*priv), GFP_KERNEL);
if (!priv) {
.irq_preinstall = msm_irq_preinstall,
.irq_postinstall = msm_irq_postinstall,
.irq_uninstall = msm_irq_uninstall,
- .get_vblank_counter = drm_vblank_no_hw_counter,
.enable_vblank = msm_enable_vblank,
.disable_vblank = msm_disable_vblank,
.gem_free_object = msm_gem_free_object,
.gem_prime_mmap = msm_gem_prime_mmap,
#ifdef CONFIG_DEBUG_FS
.debugfs_init = msm_debugfs_init,
- .debugfs_cleanup = msm_debugfs_cleanup,
#endif
.ioctls = msm_ioctls,
.num_ioctls = DRM_MSM_NUM_IOCTLS,
void msm_framebuffer_describe(struct drm_framebuffer *fb, struct seq_file *m);
int msm_debugfs_late_init(struct drm_device *dev);
int msm_rd_debugfs_init(struct drm_minor *minor);
-void msm_rd_debugfs_cleanup(struct drm_minor *minor);
+void msm_rd_debugfs_cleanup(struct msm_drm_private *priv);
void msm_rd_dump_submit(struct msm_gem_submit *submit);
int msm_perf_debugfs_init(struct drm_minor *minor);
-void msm_perf_debugfs_cleanup(struct drm_minor *minor);
+void msm_perf_debugfs_cleanup(struct msm_drm_private *priv);
#else
static inline int msm_debugfs_late_init(struct drm_device *dev) { return 0; }
static inline void msm_rd_dump_submit(struct msm_gem_submit *submit) {}
+static inline void msm_rd_debugfs_cleanup(struct msm_drm_private *priv) {}
+static inline void msm_perf_debugfs_cleanup(struct msm_drm_private *priv) {}
#endif
struct clk *msm_clk_get(struct platform_device *pdev, const char *name);
DBG();
drm_fb_helper_unregister_fbi(helper);
- drm_fb_helper_release_fbi(helper);
drm_fb_helper_fini(helper);
#ifdef CONFIG_DEBUG_FS
/* debugfs: */
int (*debugfs_init)(struct msm_kms *kms, struct drm_minor *minor);
- void (*debugfs_cleanup)(struct msm_kms *kms, struct drm_minor *minor);
#endif
};
int buftot, bufpos;
unsigned long next_jiffies;
-
- struct dentry *ent;
- struct drm_info_node *node;
};
#define SAMPLE_TIME (HZ/4)
{
struct msm_drm_private *priv = minor->dev->dev_private;
struct msm_perf_state *perf;
+ struct dentry *ent;
/* only create on first minor: */
if (priv->perf)
mutex_init(&perf->read_lock);
priv->perf = perf;
- perf->node = kzalloc(sizeof(*perf->node), GFP_KERNEL);
- if (!perf->node)
- goto fail;
-
- perf->ent = debugfs_create_file("perf", S_IFREG | S_IRUGO,
+ ent = debugfs_create_file("perf", S_IFREG | S_IRUGO,
minor->debugfs_root, perf, &perf_debugfs_fops);
- if (!perf->ent) {
+ if (!ent) {
DRM_ERROR("Cannot create /sys/kernel/debug/dri/%pd/perf\n",
minor->debugfs_root);
goto fail;
}
- perf->node->minor = minor;
- perf->node->dent = perf->ent;
- perf->node->info_ent = NULL;
-
- mutex_lock(&minor->debugfs_lock);
- list_add(&perf->node->list, &minor->debugfs_list);
- mutex_unlock(&minor->debugfs_lock);
-
return 0;
fail:
- msm_perf_debugfs_cleanup(minor);
+ msm_perf_debugfs_cleanup(priv);
return -1;
}
-void msm_perf_debugfs_cleanup(struct drm_minor *minor)
+void msm_perf_debugfs_cleanup(struct msm_drm_private *priv)
{
- struct msm_drm_private *priv = minor->dev->dev_private;
struct msm_perf_state *perf = priv->perf;
if (!perf)
priv->perf = NULL;
- debugfs_remove(perf->ent);
-
- if (perf->node) {
- mutex_lock(&minor->debugfs_lock);
- list_del(&perf->node->list);
- mutex_unlock(&minor->debugfs_lock);
- kfree(perf->node);
- }
-
mutex_destroy(&perf->read_lock);
kfree(perf);
bool open;
- struct dentry *ent;
- struct drm_info_node *node;
-
/* current submit to read out: */
struct msm_gem_submit *submit;
{
struct msm_drm_private *priv = minor->dev->dev_private;
struct msm_rd_state *rd;
+ struct dentry *ent;
/* only create on first minor: */
if (priv->rd)
init_waitqueue_head(&rd->fifo_event);
- rd->node = kzalloc(sizeof(*rd->node), GFP_KERNEL);
- if (!rd->node)
- goto fail;
-
- rd->ent = debugfs_create_file("rd", S_IFREG | S_IRUGO,
+ ent = debugfs_create_file("rd", S_IFREG | S_IRUGO,
minor->debugfs_root, rd, &rd_debugfs_fops);
- if (!rd->ent) {
+ if (!ent) {
DRM_ERROR("Cannot create /sys/kernel/debug/dri/%pd/rd\n",
minor->debugfs_root);
goto fail;
}
- rd->node->minor = minor;
- rd->node->dent = rd->ent;
- rd->node->info_ent = NULL;
-
- mutex_lock(&minor->debugfs_lock);
- list_add(&rd->node->list, &minor->debugfs_list);
- mutex_unlock(&minor->debugfs_lock);
-
return 0;
fail:
- msm_rd_debugfs_cleanup(minor);
+ msm_rd_debugfs_cleanup(priv);
return -1;
}
-void msm_rd_debugfs_cleanup(struct drm_minor *minor)
+void msm_rd_debugfs_cleanup(struct msm_drm_private *priv)
{
- struct msm_drm_private *priv = minor->dev->dev_private;
struct msm_rd_state *rd = priv->rd;
if (!rd)
return;
priv->rd = NULL;
-
- debugfs_remove(rd->ent);
-
- if (rd->node) {
- mutex_lock(&minor->debugfs_lock);
- list_del(&rd->node->list);
- mutex_unlock(&minor->debugfs_lock);
- kfree(rd->node);
- }
-
mutex_destroy(&rd->read_lock);
-
kfree(rd);
}
return drm_fb_cma_prepare_fb(&pipe->plane, plane_state);
}
-struct drm_simple_display_pipe_funcs mxsfb_funcs = {
+static struct drm_simple_display_pipe_funcs mxsfb_funcs = {
.enable = mxsfb_pipe_enable,
.disable = mxsfb_pipe_disable,
.update = mxsfb_pipe_update,
mxsfb->fbdev = drm_fbdev_cma_init(drm, 32,
drm->mode_config.num_connector);
if (IS_ERR(mxsfb->fbdev)) {
+ ret = PTR_ERR(mxsfb->fbdev);
mxsfb->fbdev = NULL;
dev_err(drm->dev, "Failed to init FB CMA area\n");
goto err_cma;
return IRQ_HANDLED;
}
-static const struct file_operations fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
-#ifdef CONFIG_COMPAT
- .compat_ioctl = drm_compat_ioctl,
-#endif
- .poll = drm_poll,
- .read = drm_read,
- .llseek = noop_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(fops);
static struct drm_driver mxsfb_driver = {
.driver_features = DRIVER_GEM | DRIVER_MODESET |
.irq_handler = mxsfb_irq_handler,
.irq_preinstall = mxsfb_irq_preinstall,
.irq_uninstall = mxsfb_irq_preinstall,
- .get_vblank_counter = drm_vblank_no_hw_counter,
.enable_vblank = mxsfb_enable_vblank,
.disable_vblank = mxsfb_disable_vblank,
.gem_free_object = drm_gem_cma_free_object,
#define MAXWELL_B /* cl9097.h */ 0x0000b197
#define PASCAL_A /* cl9097.h */ 0x0000c097
+#define PASCAL_B /* cl9097.h */ 0x0000c197
#define NV74_BSP 0x000074b0
#define MAXWELL_COMPUTE_A 0x0000b0c0
#define MAXWELL_COMPUTE_B 0x0000b1c0
#define PASCAL_COMPUTE_A 0x0000c0c0
+#define PASCAL_COMPUTE_B 0x0000c1c0
#define NV74_CIPHER 0x000074c1
#endif
NVKM_ENGINE_NVDEC,
NVKM_ENGINE_PM,
NVKM_ENGINE_SEC,
+ NVKM_ENGINE_SEC2,
NVKM_ENGINE_SW,
NVKM_ENGINE_VIC,
NVKM_ENGINE_VP,
struct nvkm_engine *msppp;
struct nvkm_engine *msvld;
struct nvkm_engine *nvenc[3];
- struct nvkm_engine *nvdec;
+ struct nvkm_nvdec *nvdec;
struct nvkm_pm *pm;
struct nvkm_engine *sec;
+ struct nvkm_sec2 *sec2;
struct nvkm_sw *sw;
struct nvkm_engine *vic;
struct nvkm_engine *vp;
int (*msppp )(struct nvkm_device *, int idx, struct nvkm_engine **);
int (*msvld )(struct nvkm_device *, int idx, struct nvkm_engine **);
int (*nvenc[3])(struct nvkm_device *, int idx, struct nvkm_engine **);
- int (*nvdec )(struct nvkm_device *, int idx, struct nvkm_engine **);
+ int (*nvdec )(struct nvkm_device *, int idx, struct nvkm_nvdec **);
int (*pm )(struct nvkm_device *, int idx, struct nvkm_pm **);
int (*sec )(struct nvkm_device *, int idx, struct nvkm_engine **);
+ int (*sec2 )(struct nvkm_device *, int idx, struct nvkm_sec2 **);
int (*sw )(struct nvkm_device *, int idx, struct nvkm_sw **);
int (*vic )(struct nvkm_device *, int idx, struct nvkm_engine **);
int (*vp )(struct nvkm_device *, int idx, struct nvkm_engine **);
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef __NVKM_CORE_MSGQUEUE_H
+#define __NVKM_CORE_MSGQUEUE_H
+
+#include <core/os.h>
+
+struct nvkm_falcon;
+struct nvkm_msgqueue;
+enum nvkm_secboot_falcon;
+
+/* Hopefully we will never have firmware arguments larger than that... */
+#define NVKM_MSGQUEUE_CMDLINE_SIZE 0x100
+
+int nvkm_msgqueue_new(u32, struct nvkm_falcon *, struct nvkm_msgqueue **);
+void nvkm_msgqueue_del(struct nvkm_msgqueue **);
+void nvkm_msgqueue_recv(struct nvkm_msgqueue *);
+int nvkm_msgqueue_reinit(struct nvkm_msgqueue *);
+
+/* useful if we run a NVIDIA-signed firmware */
+void nvkm_msgqueue_write_cmdline(struct nvkm_msgqueue *, void *);
+
+/* interface to ACR unit running on falcon (NVIDIA signed firmware) */
+int nvkm_msgqueue_acr_boot_falcon(struct nvkm_msgqueue *,
+ enum nvkm_secboot_falcon);
+
+#endif
FALCON_DMAIDX_PHYS_VID = 2,
FALCON_DMAIDX_PHYS_SYS_COH = 3,
FALCON_DMAIDX_PHYS_SYS_NCOH = 4,
+ FALCON_SEC2_DMAIDX_UCODE = 6,
};
struct nvkm_falcon {
u32 addr;
struct mutex mutex;
+ struct mutex dmem_mutex;
const struct nvkm_subdev *user;
u8 version;
u8 secret;
bool debug;
+ bool has_emem;
struct nvkm_memory *core;
bool external;
struct nvkm_engine engine;
};
+/* This constructor must be called from the owner's oneinit() hook and
+ * *not* its constructor. This is to ensure that DEVINIT has been
+ * completed, and that the device is correctly enabled before we touch
+ * falcon registers.
+ */
int nvkm_falcon_v1_new(struct nvkm_subdev *owner, const char *name, u32 addr,
struct nvkm_falcon **);
+
void nvkm_falcon_del(struct nvkm_falcon **);
int nvkm_falcon_get(struct nvkm_falcon *, const struct nvkm_subdev *);
void nvkm_falcon_put(struct nvkm_falcon *, const struct nvkm_subdev *);
int gm200_gr_new(struct nvkm_device *, int, struct nvkm_gr **);
int gm20b_gr_new(struct nvkm_device *, int, struct nvkm_gr **);
int gp100_gr_new(struct nvkm_device *, int, struct nvkm_gr **);
+int gp102_gr_new(struct nvkm_device *, int, struct nvkm_gr **);
#endif
#ifndef __NVKM_NVDEC_H__
#define __NVKM_NVDEC_H__
+#define nvkm_nvdec(p) container_of((p), struct nvkm_nvdec, engine)
#include <core/engine.h>
+
+struct nvkm_nvdec {
+ struct nvkm_engine engine;
+ struct nvkm_falcon *falcon;
+};
+
+int gp102_nvdec_new(struct nvkm_device *, int, struct nvkm_nvdec **);
#endif
--- /dev/null
+#ifndef __NVKM_SEC2_H__
+#define __NVKM_SEC2_H__
+#include <core/engine.h>
+
+struct nvkm_sec2 {
+ struct nvkm_engine engine;
+ struct nvkm_falcon *falcon;
+ struct nvkm_msgqueue *queue;
+ struct work_struct work;
+};
+
+int gp102_sec2_new(struct nvkm_device *, int, struct nvkm_sec2 **);
+#endif
int mcp77_fb_new(struct nvkm_device *, int, struct nvkm_fb **);
int mcp89_fb_new(struct nvkm_device *, int, struct nvkm_fb **);
int gf100_fb_new(struct nvkm_device *, int, struct nvkm_fb **);
+int gf108_fb_new(struct nvkm_device *, int, struct nvkm_fb **);
int gk104_fb_new(struct nvkm_device *, int, struct nvkm_fb **);
int gk20a_fb_new(struct nvkm_device *, int, struct nvkm_fb **);
int gm107_fb_new(struct nvkm_device *, int, struct nvkm_fb **);
};
struct nvkm_ram_func {
+ u64 upper;
+ u32 (*probe_fbp)(const struct nvkm_ram_func *, struct nvkm_device *,
+ int fbp, int *pltcs);
+ u32 (*probe_fbp_amount)(const struct nvkm_ram_func *, u32 fbpao,
+ struct nvkm_device *, int fbp, int *pltcs);
+ u32 (*probe_fbpa_amount)(struct nvkm_device *, int fbpa);
void *(*dtor)(struct nvkm_ram *);
int (*init)(struct nvkm_ram *);
int nvkm_i2c_aux_acquire(struct nvkm_i2c_aux *);
void nvkm_i2c_aux_release(struct nvkm_i2c_aux *);
int nvkm_i2c_aux_xfer(struct nvkm_i2c_aux *, bool retry, u8 type,
- u32 addr, u8 *data, u8 size);
+ u32 addr, u8 *data, u8 *size);
int nvkm_i2c_aux_lnk_ctl(struct nvkm_i2c_aux *, int link_nr, int link_bw,
bool enhanced_framing);
static inline int
nvkm_rdaux(struct nvkm_i2c_aux *aux, u32 addr, u8 *data, u8 size)
{
+ const u8 xfer = size;
int ret = nvkm_i2c_aux_acquire(aux);
if (ret == 0) {
- ret = nvkm_i2c_aux_xfer(aux, true, 9, addr, data, size);
+ ret = nvkm_i2c_aux_xfer(aux, true, 9, addr, data, &size);
+ WARN_ON(!ret && size != xfer);
nvkm_i2c_aux_release(aux);
}
return ret;
{
int ret = nvkm_i2c_aux_acquire(aux);
if (ret == 0) {
- ret = nvkm_i2c_aux_xfer(aux, true, 8, addr, data, size);
+ ret = nvkm_i2c_aux_xfer(aux, true, 8, addr, data, &size);
nvkm_i2c_aux_release(aux);
}
return ret;
const struct nvkm_pmu_func *func;
struct nvkm_subdev subdev;
struct nvkm_falcon *falcon;
+ struct nvkm_msgqueue *queue;
struct {
u32 base;
NVKM_SECBOOT_FALCON_RESERVED = 1,
NVKM_SECBOOT_FALCON_FECS = 2,
NVKM_SECBOOT_FALCON_GPCCS = 3,
- NVKM_SECBOOT_FALCON_END = 4,
+ NVKM_SECBOOT_FALCON_SEC2 = 7,
+ NVKM_SECBOOT_FALCON_END = 8,
NVKM_SECBOOT_FALCON_INVALID = 0xffffffff,
};
+extern const char *nvkm_secboot_falcon_name[];
+
/**
* @wpr_set: whether the WPR region is currently set
*/
struct nvkm_acr *acr;
struct nvkm_subdev subdev;
struct nvkm_falcon *boot_falcon;
+ struct nvkm_falcon *halt_falcon;
u64 wpr_addr;
u32 wpr_size;
int gm200_secboot_new(struct nvkm_device *, int, struct nvkm_secboot **);
int gm20b_secboot_new(struct nvkm_device *, int, struct nvkm_secboot **);
+int gp102_secboot_new(struct nvkm_device *, int, struct nvkm_secboot **);
#endif
nouveau_dsm_priv.dhandle = dhandle;
acpi_get_name(nouveau_dsm_priv.dhandle, ACPI_FULL_PATHNAME,
&buffer);
- printk(KERN_INFO "VGA switcheroo: detected Optimus DSM method %s handle\n",
+ pr_info("VGA switcheroo: detected Optimus DSM method %s handle\n",
acpi_method_name);
if (has_power_resources)
pr_info("nouveau: detected PR support, will not use DSM\n");
nouveau_dsm_priv.dhandle = dhandle;
acpi_get_name(nouveau_dsm_priv.dhandle, ACPI_FULL_PATHNAME,
&buffer);
- printk(KERN_INFO "VGA switcheroo: detected DSM switching method %s handle\n",
+ pr_info("VGA switcheroo: detected DSM switching method %s handle\n",
acpi_method_name);
nouveau_dsm_priv.dsm_detected = true;
ret = true;
status = acpi_evaluate_object(rom_handle, NULL, &rom_arg, &buffer);
if (ACPI_FAILURE(status)) {
- printk(KERN_INFO "failed to evaluate ROM got %s\n", acpi_format_exception(status));
+ pr_info("failed to evaluate ROM got %s\n",
+ acpi_format_exception(status));
return -ENODEV;
}
obj = (union acpi_object *)buffer.pointer;
container_of(obj, typeof(*nv_connector), aux);
struct nouveau_encoder *nv_encoder;
struct nvkm_i2c_aux *aux;
+ u8 size = msg->size;
int ret;
nv_encoder = find_encoder(&nv_connector->base, DCB_OUTPUT_DP);
return ret;
ret = nvkm_i2c_aux_xfer(aux, false, msg->request, msg->address,
- msg->buffer, msg->size);
+ msg->buffer, &size);
nvkm_i2c_aux_release(aux);
if (ret >= 0) {
msg->reply = ret;
- return msg->size;
+ return size;
}
return ret;
static int
nouveau_debugfs_pstate_get(struct seq_file *m, void *data)
{
- struct drm_info_node *node = (struct drm_info_node *) m->private;
- struct nouveau_debugfs *debugfs = nouveau_debugfs(node->minor->dev);
+ struct drm_device *drm = m->private;
+ struct nouveau_debugfs *debugfs = nouveau_debugfs(drm);
struct nvif_object *ctrl = &debugfs->ctrl;
struct nvif_control_pstate_info_v0 info = {};
int ret, i;
size_t len, loff_t *offp)
{
struct seq_file *m = file->private_data;
- struct drm_info_node *node = (struct drm_info_node *) m->private;
- struct nouveau_debugfs *debugfs = nouveau_debugfs(node->minor->dev);
+ struct drm_device *drm = m->private;
+ struct nouveau_debugfs *debugfs = nouveau_debugfs(drm);
struct nvif_object *ctrl = &debugfs->ctrl;
struct nvif_control_pstate_user_v0 args = { .pwrsrc = -EINVAL };
char buf[32] = {}, *tmp, *cur = buf;
{"pstate", &nouveau_pstate_fops},
};
-static int
-nouveau_debugfs_create_file(struct drm_minor *minor,
- const struct nouveau_debugfs_files *ndf)
-{
- struct drm_info_node *node;
-
- node = kmalloc(sizeof(*node), GFP_KERNEL);
- if (node == NULL)
- return -ENOMEM;
-
- node->minor = minor;
- node->info_ent = (const void *)ndf->fops;
- node->dent = debugfs_create_file(ndf->name, S_IRUGO | S_IWUSR,
- minor->debugfs_root, node, ndf->fops);
- if (!node->dent) {
- kfree(node);
- return -ENOMEM;
- }
-
- mutex_lock(&minor->debugfs_lock);
- list_add(&node->list, &minor->debugfs_list);
- mutex_unlock(&minor->debugfs_lock);
- return 0;
-}
-
int
nouveau_drm_debugfs_init(struct drm_minor *minor)
{
- int i, ret;
+ struct dentry *dentry;
+ int i;
for (i = 0; i < ARRAY_SIZE(nouveau_debugfs_files); i++) {
- ret = nouveau_debugfs_create_file(minor,
- &nouveau_debugfs_files[i]);
-
- if (ret)
- return ret;
+ dentry = debugfs_create_file(nouveau_debugfs_files[i].name,
+ S_IRUGO | S_IWUSR,
+ minor->debugfs_root, minor->dev,
+ nouveau_debugfs_files[i].fops);
+ if (!dentry)
+ return -ENOMEM;
}
return drm_debugfs_create_files(nouveau_debugfs_list,
minor->debugfs_root, minor);
}
-void
-nouveau_drm_debugfs_cleanup(struct drm_minor *minor)
-{
- int i;
-
- drm_debugfs_remove_files(nouveau_debugfs_list, NOUVEAU_DEBUGFS_ENTRIES,
- minor);
-
- for (i = 0; i < ARRAY_SIZE(nouveau_debugfs_files); i++) {
- drm_debugfs_remove_files((struct drm_info_list *)
- nouveau_debugfs_files[i].fops,
- 1, minor);
- }
-}
-
int
nouveau_debugfs_init(struct nouveau_drm *drm)
{
}
extern int nouveau_drm_debugfs_init(struct drm_minor *);
-extern void nouveau_drm_debugfs_cleanup(struct drm_minor *);
extern int nouveau_debugfs_init(struct nouveau_drm *);
extern void nouveau_debugfs_fini(struct nouveau_drm *);
#else
return 0;
}
-static inline void
-nouveau_drm_debugfs_cleanup(struct drm_minor *minor)
-{
-}
-
static inline int
nouveau_debugfs_init(struct nouveau_drm *drm)
{
kfree(disp);
}
-static int
-nouveau_atomic_disable_connector(struct drm_atomic_state *state,
- struct drm_connector *connector)
-{
- struct drm_connector_state *connector_state;
- struct drm_crtc *crtc;
- struct drm_crtc_state *crtc_state;
- struct drm_plane_state *plane_state;
- struct drm_plane *plane;
- int ret;
-
- if (!(crtc = connector->state->crtc))
- return 0;
-
- connector_state = drm_atomic_get_connector_state(state, connector);
- if (IS_ERR(connector_state))
- return PTR_ERR(connector_state);
-
- ret = drm_atomic_set_crtc_for_connector(connector_state, NULL);
- if (ret)
- return ret;
-
- crtc_state = drm_atomic_get_crtc_state(state, crtc);
- if (IS_ERR(crtc_state))
- return PTR_ERR(crtc_state);
-
- ret = drm_atomic_set_mode_for_crtc(crtc_state, NULL);
- if (ret)
- return ret;
-
- crtc_state->active = false;
-
- drm_for_each_plane_mask(plane, connector->dev, crtc_state->plane_mask) {
- plane_state = drm_atomic_get_plane_state(state, plane);
- if (IS_ERR(plane_state))
- return PTR_ERR(plane_state);
-
- ret = drm_atomic_set_crtc_for_plane(plane_state, NULL);
- if (ret)
- return ret;
-
- drm_atomic_set_fb_for_plane(plane_state, NULL);
- }
-
- return 0;
-}
-
-static int
-nouveau_atomic_disable(struct drm_device *dev,
- struct drm_modeset_acquire_ctx *ctx)
-{
- struct drm_atomic_state *state;
- struct drm_connector *connector;
- int ret;
-
- state = drm_atomic_state_alloc(dev);
- if (!state)
- return -ENOMEM;
-
- state->acquire_ctx = ctx;
-
- drm_for_each_connector(connector, dev) {
- ret = nouveau_atomic_disable_connector(state, connector);
- if (ret)
- break;
- }
-
- if (ret == 0)
- ret = drm_atomic_commit(state);
- drm_atomic_state_put(state);
- return ret;
-}
-
-static struct drm_atomic_state *
-nouveau_atomic_suspend(struct drm_device *dev)
-{
- struct drm_modeset_acquire_ctx ctx;
- struct drm_atomic_state *state;
- int ret;
-
- drm_modeset_acquire_init(&ctx, 0);
-
-retry:
- ret = drm_modeset_lock_all_ctx(dev, &ctx);
- if (ret < 0) {
- state = ERR_PTR(ret);
- goto unlock;
- }
-
- state = drm_atomic_helper_duplicate_state(dev, &ctx);
- if (IS_ERR(state))
- goto unlock;
-
- ret = nouveau_atomic_disable(dev, &ctx);
- if (ret < 0) {
- drm_atomic_state_put(state);
- state = ERR_PTR(ret);
- goto unlock;
- }
-
-unlock:
- if (PTR_ERR(state) == -EDEADLK) {
- drm_modeset_backoff(&ctx);
- goto retry;
- }
-
- drm_modeset_drop_locks(&ctx);
- drm_modeset_acquire_fini(&ctx);
- return state;
-}
-
int
nouveau_display_suspend(struct drm_device *dev, bool runtime)
{
if (drm_drv_uses_atomic_modeset(dev)) {
if (!runtime) {
- disp->suspend = nouveau_atomic_suspend(dev);
+ disp->suspend = drm_atomic_helper_suspend(dev);
if (IS_ERR(disp->suspend)) {
int ret = PTR_ERR(disp->suspend);
disp->suspend = NULL;
if (dev->pdev)
return nouveau_pci_name(dev->pdev);
else
- return nouveau_platform_name(dev->platformdev);
+ return nouveau_platform_name(to_platform_device(dev->dev));
}
static void
#if defined(CONFIG_DEBUG_FS)
.debugfs_init = nouveau_drm_debugfs_init,
- .debugfs_cleanup = nouveau_drm_debugfs_cleanup,
#endif
- .get_vblank_counter = drm_vblank_no_hw_counter,
.enable_vblank = nouveau_display_vblank_enable,
.disable_vblank = nouveau_display_vblank_disable,
.get_scanout_position = nouveau_display_scanoutpos,
goto err_free;
}
- drm->platformdev = pdev;
platform_set_drvdata(pdev, drm);
return drm;
struct nouveau_framebuffer *nouveau_fb = nouveau_framebuffer(fbcon->helper.fb);
drm_fb_helper_unregister_fbi(&fbcon->helper);
- drm_fb_helper_release_fbi(&fbcon->helper);
drm_fb_helper_fini(&fbcon->helper);
if (nouveau_fb->nvbo) {
goto done;
ntfy->p->base.event = &ntfy->p->e.base;
ntfy->p->base.file_priv = f;
- ntfy->p->base.pid = current->pid;
ntfy->p->e.base.type = DRM_NOUVEAU_EVENT_NVIF;
ntfy->p->e.base.length = sizeof(ntfy->p->e.base) + ntfy->reply;
return;
if (state == VGA_SWITCHEROO_ON) {
- printk(KERN_ERR "VGA switcheroo: switched nouveau on\n");
+ pr_err("VGA switcheroo: switched nouveau on\n");
dev->switch_power_state = DRM_SWITCH_POWER_CHANGING;
nouveau_pmops_resume(&pdev->dev);
drm_kms_helper_poll_enable(dev);
dev->switch_power_state = DRM_SWITCH_POWER_ON;
} else {
- printk(KERN_ERR "VGA switcheroo: switched nouveau off\n");
+ pr_err("VGA switcheroo: switched nouveau off\n");
dev->switch_power_state = DRM_SWITCH_POWER_CHANGING;
drm_kms_helper_poll_disable(dev);
nouveau_switcheroo_optimus_dsm();
break;
) < 0) {
mutex_unlock(&dmac->lock);
- printk(KERN_ERR "nouveau: evo channel stalled\n");
+ pr_err("nouveau: evo channel stalled\n");
return NULL;
}
mutex_unlock(&dmac->lock);
}
-#define evo_mthd(p,m,s) do { \
- const u32 _m = (m), _s = (s); \
- if (drm_debug & DRM_UT_KMS) \
- printk(KERN_ERR "%04x %d %s\n", _m, _s, __func__); \
- *((p)++) = ((_s << 18) | _m); \
+#define evo_mthd(p, m, s) do { \
+ const u32 _m = (m), _s = (s); \
+ if (drm_debug & DRM_UT_KMS) \
+ pr_err("%04x %d %s\n", _m, _s, __func__); \
+ *((p)++) = ((_s << 18) | _m); \
} while(0)
-#define evo_data(p,d) do { \
- const u32 _d = (d); \
- if (drm_debug & DRM_UT_KMS) \
- printk(KERN_ERR "\t%08x\n", _d); \
- *((p)++) = _d; \
+#define evo_data(p, d) do { \
+ const u32 _d = (d); \
+ if (drm_debug & DRM_UT_KMS) \
+ pr_err("\t%08x\n", _d); \
+ *((p)++) = _d; \
} while(0)
/******************************************************************************
static int
nv50_wndw_atomic_check_acquire(struct nv50_wndw *wndw,
struct nv50_wndw_atom *asyw,
- struct nv50_head_atom *asyh)
+ struct nv50_head_atom *asyh,
+ u32 pflip_flags)
{
struct nouveau_framebuffer *fb = nouveau_framebuffer(asyw->state.fb);
struct nouveau_drm *drm = nouveau_drm(wndw->plane.dev);
asyw->image.w = fb->base.width;
asyw->image.h = fb->base.height;
asyw->image.kind = (fb->nvbo->tile_flags & 0x0000ff00) >> 8;
+
+ asyw->interval = pflip_flags & DRM_MODE_PAGE_FLIP_ASYNC ? 0 : 1;
+
if (asyw->image.kind) {
asyw->image.layout = 0;
if (drm->client.device.info.chipset >= 0xc0)
struct nv50_head_atom *harm = NULL, *asyh = NULL;
bool varm = false, asyv = false, asym = false;
int ret;
+ u32 pflip_flags = 0;
NV_ATOMIC(drm, "%s atomic_check\n", plane->name);
if (asyw->state.crtc) {
return PTR_ERR(asyh);
asym = drm_atomic_crtc_needs_modeset(&asyh->state);
asyv = asyh->state.active;
+ pflip_flags = asyh->state.pageflip_flags;
}
if (armw->state.crtc) {
asyw->set.point = true;
if (!varm || asym || armw->state.fb != asyw->state.fb) {
- ret = nv50_wndw_atomic_check_acquire(wndw, asyw, asyh);
+ ret = nv50_wndw_atomic_check_acquire(
+ wndw, asyw, asyh, pflip_flags);
if (ret)
return ret;
}
.atomic_check = nv50_head_atomic_check,
};
-/* This is identical to the version in the atomic helpers, except that
- * it supports non-vblanked ("async") page flips.
- */
-static int
-nv50_head_page_flip(struct drm_crtc *crtc, struct drm_framebuffer *fb,
- struct drm_pending_vblank_event *event, u32 flags)
-{
- struct drm_plane *plane = crtc->primary;
- struct drm_atomic_state *state;
- struct drm_plane_state *plane_state;
- struct drm_crtc_state *crtc_state;
- int ret = 0;
-
- state = drm_atomic_state_alloc(plane->dev);
- if (!state)
- return -ENOMEM;
-
- state->acquire_ctx = drm_modeset_legacy_acquire_ctx(crtc);
-retry:
- crtc_state = drm_atomic_get_crtc_state(state, crtc);
- if (IS_ERR(crtc_state)) {
- ret = PTR_ERR(crtc_state);
- goto fail;
- }
- crtc_state->event = event;
-
- plane_state = drm_atomic_get_plane_state(state, plane);
- if (IS_ERR(plane_state)) {
- ret = PTR_ERR(plane_state);
- goto fail;
- }
-
- ret = drm_atomic_set_crtc_for_plane(plane_state, crtc);
- if (ret != 0)
- goto fail;
- drm_atomic_set_fb_for_plane(plane_state, fb);
-
- /* Make sure we don't accidentally do a full modeset. */
- state->allow_modeset = false;
- if (!crtc_state->active) {
- DRM_DEBUG_ATOMIC("[CRTC:%d] disabled, rejecting legacy flip\n",
- crtc->base.id);
- ret = -EINVAL;
- goto fail;
- }
-
- if (flags & DRM_MODE_PAGE_FLIP_ASYNC)
- nv50_wndw_atom(plane_state)->interval = 0;
-
- ret = drm_atomic_nonblocking_commit(state);
-fail:
- if (ret == -EDEADLK)
- goto backoff;
-
- drm_atomic_state_put(state);
- return ret;
-
-backoff:
- drm_atomic_state_clear(state);
- drm_atomic_legacy_backoff(state);
-
- /*
- * Someone might have exchanged the framebuffer while we dropped locks
- * in the backoff code. We need to fix up the fb refcount tracking the
- * core does for us.
- */
- plane->old_fb = plane->fb;
-
- goto retry;
-}
-
static int
nv50_head_gamma_set(struct drm_crtc *crtc, u16 *r, u16 *g, u16 *b,
uint32_t size)
.gamma_set = nv50_head_gamma_set,
.destroy = nv50_head_destroy,
.set_config = drm_atomic_helper_set_config,
- .page_flip = nv50_head_page_flip,
+ .page_flip = drm_atomic_helper_page_flip,
.set_property = drm_atomic_helper_crtc_set_property,
.atomic_duplicate_state = nv50_head_atomic_duplicate_state,
.atomic_destroy_state = nv50_head_atomic_destroy_state,
struct nvkm_i2c_aux *aux =
nvkm_i2c_aux_find(i2c, dcbe->i2c_index);
if (aux) {
- nv_encoder->i2c = &aux->i2c;
+ nv_encoder->i2c = &nv_connector->aux.ddc;
nv_encoder->aux = aux;
}
static int
nv50_pior_create(struct drm_connector *connector, struct dcb_output *dcbe)
{
+ struct nouveau_connector *nv_connector = nouveau_connector(connector);
struct nouveau_drm *drm = nouveau_drm(connector->dev);
struct nvkm_i2c *i2c = nvxx_i2c(&drm->client.device);
struct nvkm_i2c_bus *bus = NULL;
break;
case DCB_OUTPUT_DP:
aux = nvkm_i2c_aux_find(i2c, NVKM_I2C_AUX_EXT(dcbe->extdev));
- ddc = aux ? &aux->i2c : NULL;
+ ddc = aux ? &nv_connector->aux.ddc : NULL;
type = DRM_MODE_ENCODER_TMDS;
break;
default:
{
struct nvkm_mm_node *node;
- printk(KERN_ERR "nvkm: %s\n", header);
- printk(KERN_ERR "nvkm: node list:\n");
+ pr_err("nvkm: %s\n", header);
+ pr_err("nvkm: node list:\n");
list_for_each_entry(node, &mm->nodes, nl_entry) {
- printk(KERN_ERR "nvkm: \t%08x %08x %d\n",
+ pr_err("nvkm: \t%08x %08x %d\n",
node->offset, node->length, node->type);
}
- printk(KERN_ERR "nvkm: free list:\n");
+ pr_err("nvkm: free list:\n");
list_for_each_entry(node, &mm->free, fl_entry) {
- printk(KERN_ERR "nvkm: \t%08x %08x %d\n",
+ pr_err("nvkm: \t%08x %08x %d\n",
node->offset, node->length, node->type);
}
}
[NVKM_ENGINE_NVDEC ] = "nvdec",
[NVKM_ENGINE_PM ] = "pm",
[NVKM_ENGINE_SEC ] = "sec",
+ [NVKM_ENGINE_SEC2 ] = "sec2",
[NVKM_ENGINE_SW ] = "sw",
[NVKM_ENGINE_VIC ] = "vic",
[NVKM_ENGINE_VP ] = "vp",
include $(src)/nvkm/engine/nvdec/Kbuild
include $(src)/nvkm/engine/pm/Kbuild
include $(src)/nvkm/engine/sec/Kbuild
+include $(src)/nvkm/engine/sec2/Kbuild
include $(src)/nvkm/engine/sw/Kbuild
include $(src)/nvkm/engine/vic/Kbuild
include $(src)/nvkm/engine/vp/Kbuild
.bus = gf100_bus_new,
.clk = gf100_clk_new,
.devinit = gf100_devinit_new,
- .fb = gf100_fb_new,
+ .fb = gf108_fb_new,
.fuse = gf100_fuse_new,
.gpio = g94_gpio_new,
.i2c = g94_i2c_new,
.ltc = gp100_ltc_new,
.mc = gp100_mc_new,
.mmu = gf100_mmu_new,
+ .secboot = gp102_secboot_new,
+ .sec2 = gp102_sec2_new,
+ .nvdec = gp102_nvdec_new,
.pci = gp100_pci_new,
.pmu = gp102_pmu_new,
.timer = gk20a_timer_new,
.disp = gp102_disp_new,
.dma = gf119_dma_new,
.fifo = gp100_fifo_new,
+ .gr = gp102_gr_new,
+ .sw = gf100_sw_new,
};
static const struct nvkm_device_chip
.ltc = gp100_ltc_new,
.mc = gp100_mc_new,
.mmu = gf100_mmu_new,
+ .secboot = gp102_secboot_new,
+ .sec2 = gp102_sec2_new,
+ .nvdec = gp102_nvdec_new,
.pci = gp100_pci_new,
.pmu = gp102_pmu_new,
.timer = gk20a_timer_new,
.disp = gp102_disp_new,
.dma = gf119_dma_new,
.fifo = gp100_fifo_new,
+ .gr = gp102_gr_new,
+ .sw = gf100_sw_new,
};
static const struct nvkm_device_chip
.ltc = gp100_ltc_new,
.mc = gp100_mc_new,
.mmu = gf100_mmu_new,
+ .secboot = gp102_secboot_new,
+ .sec2 = gp102_sec2_new,
+ .nvdec = gp102_nvdec_new,
.pci = gp100_pci_new,
.pmu = gp102_pmu_new,
.timer = gk20a_timer_new,
.disp = gp102_disp_new,
.dma = gf119_dma_new,
.fifo = gp100_fifo_new,
+ .gr = gp102_gr_new,
+ .sw = gf100_sw_new,
};
static int
_(NVENC0 , device->nvenc[0], device->nvenc[0]);
_(NVENC1 , device->nvenc[1], device->nvenc[1]);
_(NVENC2 , device->nvenc[2], device->nvenc[2]);
- _(NVDEC , device->nvdec , device->nvdec);
+ _(NVDEC , device->nvdec , &device->nvdec->engine);
_(PM , device->pm , &device->pm->engine);
_(SEC , device->sec , device->sec);
+ _(SEC2 , device->sec2 , &device->sec2->engine);
_(SW , device->sw , &device->sw->engine);
_(VIC , device->vic , device->vic);
_(VP , device->vp , device->vp);
_(NVKM_ENGINE_NVDEC , nvdec);
_(NVKM_ENGINE_PM , pm);
_(NVKM_ENGINE_SEC , sec);
+ _(NVKM_ENGINE_SEC2 , sec2);
_(NVKM_ENGINE_SW , sw);
_(NVKM_ENGINE_VIC , vic);
_(NVKM_ENGINE_VP , vp);
#include <engine/nvdec.h>
#include <engine/pm.h>
#include <engine/sec.h>
+#include <engine/sec2.h>
#include <engine/sw.h>
#include <engine/vic.h>
#include <engine/vp.h>
nvkm-y += nvkm/engine/gr/gm200.o
nvkm-y += nvkm/engine/gr/gm20b.o
nvkm-y += nvkm/engine/gr/gp100.o
+nvkm-y += nvkm/engine/gr/gp102.o
nvkm-y += nvkm/engine/gr/ctxnv40.o
nvkm-y += nvkm/engine/gr/ctxnv50.o
nvkm-y += nvkm/engine/gr/ctxgm200.o
nvkm-y += nvkm/engine/gr/ctxgm20b.o
nvkm-y += nvkm/engine/gr/ctxgp100.o
+nvkm-y += nvkm/engine/gr/ctxgp102.o
extern const struct gf100_grctx_func gm20b_grctx;
extern const struct gf100_grctx_func gp100_grctx;
+void gp100_grctx_generate_main(struct gf100_gr *, struct gf100_grctx *);
+void gp100_grctx_generate_pagepool(struct gf100_grctx *);
+
+extern const struct gf100_grctx_func gp102_grctx;
/* context init value lists */
* PGRAPH context implementation
******************************************************************************/
-static void
+void
gp100_grctx_generate_pagepool(struct gf100_grctx *info)
{
const struct gf100_grctx_func *grctx = info->gr->func->grctx;
nvkm_wr32(device, 0x405ba0 + (i * 4), gpcs[i]);
}
-static void
+void
gp100_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
--- /dev/null
+/*
+ * Copyright 2016 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs <bskeggs@redhat.com>
+ */
+#include "ctxgf100.h"
+
+#include <subdev/fb.h>
+
+/*******************************************************************************
+ * PGRAPH context implementation
+ ******************************************************************************/
+
+static void
+gp102_grctx_generate_attrib(struct gf100_grctx *info)
+{
+ struct gf100_gr *gr = info->gr;
+ const struct gf100_grctx_func *grctx = gr->func->grctx;
+ const u32 alpha = grctx->alpha_nr;
+ const u32 attrib = grctx->attrib_nr;
+ const u32 pertpc = 0x20 * (grctx->attrib_nr_max + grctx->alpha_nr_max);
+ const u32 size = roundup(gr->tpc_total * pertpc, 0x80);
+ const u32 access = NV_MEM_ACCESS_RW;
+ const int s = 12;
+ const int b = mmio_vram(info, size, (1 << s), access);
+ const int max_batches = 0xffff;
+ u32 ao = 0;
+ u32 bo = ao + grctx->alpha_nr_max * gr->tpc_total;
+ int gpc, ppc, n = 0;
+
+ mmio_refn(info, 0x418810, 0x80000000, s, b);
+ mmio_refn(info, 0x419848, 0x10000000, s, b);
+ mmio_refn(info, 0x419c2c, 0x10000000, s, b);
+ mmio_refn(info, 0x419b00, 0x00000000, s, b);
+ mmio_wr32(info, 0x419b04, 0x80000000 | size >> 7);
+ mmio_wr32(info, 0x405830, attrib);
+ mmio_wr32(info, 0x40585c, alpha);
+ mmio_wr32(info, 0x4064c4, ((alpha / 4) << 16) | max_batches);
+
+ for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
+ for (ppc = 0; ppc < gr->ppc_nr[gpc]; ppc++, n++) {
+ const u32 as = alpha * gr->ppc_tpc_nr[gpc][ppc];
+ const u32 bs = attrib * gr->ppc_tpc_nr[gpc][ppc];
+ const u32 u = 0x418ea0 + (n * 0x04);
+ const u32 o = PPC_UNIT(gpc, ppc, 0);
+ const u32 p = GPC_UNIT(gpc, 0xc44 + (ppc * 4));
+ if (!(gr->ppc_mask[gpc] & (1 << ppc)))
+ continue;
+ mmio_wr32(info, o + 0xc0, bs);
+ mmio_wr32(info, p, bs);
+ mmio_wr32(info, o + 0xf4, bo);
+ mmio_wr32(info, o + 0xf0, bs);
+ bo += grctx->attrib_nr_max * gr->ppc_tpc_nr[gpc][ppc];
+ mmio_wr32(info, o + 0xe4, as);
+ mmio_wr32(info, o + 0xf8, ao);
+ ao += grctx->alpha_nr_max * gr->ppc_tpc_nr[gpc][ppc];
+ mmio_wr32(info, u, bs);
+ }
+ }
+
+ mmio_wr32(info, 0x4181e4, 0x00000100);
+ mmio_wr32(info, 0x41befc, 0x00000100);
+}
+
+const struct gf100_grctx_func
+gp102_grctx = {
+ .main = gp100_grctx_generate_main,
+ .unkn = gk104_grctx_generate_unkn,
+ .bundle = gm107_grctx_generate_bundle,
+ .bundle_size = 0x3000,
+ .bundle_min_gpm_fifo_depth = 0x180,
+ .bundle_token_limit = 0x900,
+ .pagepool = gp100_grctx_generate_pagepool,
+ .pagepool_size = 0x20000,
+ .attrib = gp102_grctx_generate_attrib,
+ .attrib_nr_max = 0x5d4,
+ .attrib_nr = 0x320,
+ .alpha_nr_max = 0xc00,
+ .alpha_nr = 0x800,
+};
gf100_gr_oneinit(struct nvkm_gr *base)
{
struct gf100_gr *gr = gf100_gr(base);
- struct nvkm_device *device = gr->base.engine.subdev.device;
+ struct nvkm_subdev *subdev = &gr->base.engine.subdev;
+ struct nvkm_device *device = subdev->device;
int i, j;
+ int ret;
+
+ ret = nvkm_falcon_v1_new(subdev, "FECS", 0x409000, &gr->fecs);
+ if (ret)
+ return ret;
+
+ ret = nvkm_falcon_v1_new(subdev, "GPCCS", 0x41a000, &gr->gpccs);
+ if (ret)
+ return ret;
nvkm_pmu_pgob(device->pmu, false);
gf100_gr_ctor(const struct gf100_gr_func *func, struct nvkm_device *device,
int index, struct gf100_gr *gr)
{
- struct nvkm_subdev *subdev = &gr->base.engine.subdev;
- int ret;
-
gr->func = func;
gr->firmware = nvkm_boolopt(device->cfgopt, "NvGrUseFW",
func->fecs.ucode == NULL);
- ret = nvkm_gr_ctor(&gf100_gr_, device, index,
- gr->firmware || func->fecs.ucode != NULL,
- &gr->base);
- if (ret)
- return ret;
-
- ret = nvkm_falcon_v1_new(subdev, "FECS", 0x409000, &gr->fecs);
- if (ret)
- return ret;
-
- return nvkm_falcon_v1_new(subdev, "GPCCS", 0x41a000, &gr->gpccs);
+ return nvkm_gr_ctor(&gf100_gr_, device, index,
+ gr->firmware || func->fecs.ucode != NULL,
+ &gr->base);
}
int
void (*init_gpc_mmu)(struct gf100_gr *);
void (*init_rop_active_fbps)(struct gf100_gr *);
void (*init_ppc_exceptions)(struct gf100_gr *);
+ void (*init_swdx_pes_mask)(struct gf100_gr *);
void (*set_hww_esr_report_mask)(struct gf100_gr *);
const struct gf100_gr_pack *mmio;
struct {
int gm200_gr_init(struct gf100_gr *);
int gm200_gr_rops(struct gf100_gr *);
+int gp100_gr_init(struct gf100_gr *);
+void gp100_gr_init_rop_active_fbps(struct gf100_gr *);
+
#define gf100_gr_chan(p) container_of((p), struct gf100_gr_chan, object)
struct gf100_gr_chan {
* PGRAPH engine/subdev functions
******************************************************************************/
-static void
+void
gp100_gr_init_rop_active_fbps(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
nvkm_mask(device, 0x408958, 0x0000000f, fbp_count); /* crop */
}
-static int
+int
gp100_gr_init(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
nvkm_wr32(device, GPC_BCAST(0x033c), nvkm_rd32(device, 0x100804));
gr->func->init_rop_active_fbps(gr);
+ if (gr->func->init_swdx_pes_mask)
+ gr->func->init_swdx_pes_mask(gr);
nvkm_wr32(device, 0x400500, 0x00010001);
nvkm_wr32(device, 0x400100, 0xffffffff);
--- /dev/null
+/*
+ * Copyright 2016 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs <bskeggs@redhat.com>
+ */
+#include "gf100.h"
+#include "ctxgf100.h"
+
+#include <nvif/class.h>
+
+static void
+gp102_gr_init_swdx_pes_mask(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+ u32 mask = 0, data, gpc;
+
+ for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
+ data = nvkm_rd32(device, GPC_UNIT(gpc, 0x0c50)) & 0x0000000f;
+ mask |= data << (gpc * 4);
+ }
+
+ nvkm_wr32(device, 0x4181d0, mask);
+}
+
+static const struct gf100_gr_func
+gp102_gr = {
+ .init = gp100_gr_init,
+ .init_gpc_mmu = gm200_gr_init_gpc_mmu,
+ .init_rop_active_fbps = gp100_gr_init_rop_active_fbps,
+ .init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
+ .init_swdx_pes_mask = gp102_gr_init_swdx_pes_mask,
+ .rops = gm200_gr_rops,
+ .ppc_nr = 3,
+ .grctx = &gp102_grctx,
+ .sclass = {
+ { -1, -1, FERMI_TWOD_A },
+ { -1, -1, KEPLER_INLINE_TO_MEMORY_B },
+ { -1, -1, PASCAL_B, &gf100_fermi },
+ { -1, -1, PASCAL_COMPUTE_B },
+ {}
+ }
+};
+
+int
+gp102_gr_new(struct nvkm_device *device, int index, struct nvkm_gr **pgr)
+{
+ return gm200_gr_new_(&gp102_gr, device, index, pgr);
+}
-#nvkm-y += nvkm/engine/nvdec/base.o
+nvkm-y += nvkm/engine/nvdec/base.o
+nvkm-y += nvkm/engine/nvdec/gp102.o
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+#include <engine/falcon.h>
+
+static int
+nvkm_nvdec_oneinit(struct nvkm_engine *engine)
+{
+ struct nvkm_nvdec *nvdec = nvkm_nvdec(engine);
+ return nvkm_falcon_v1_new(&nvdec->engine.subdev, "NVDEC", 0x84000,
+ &nvdec->falcon);
+}
+
+static void *
+nvkm_nvdec_dtor(struct nvkm_engine *engine)
+{
+ struct nvkm_nvdec *nvdec = nvkm_nvdec(engine);
+ nvkm_falcon_del(&nvdec->falcon);
+ return nvdec;
+}
+
+static const struct nvkm_engine_func
+nvkm_nvdec = {
+ .dtor = nvkm_nvdec_dtor,
+ .oneinit = nvkm_nvdec_oneinit,
+};
+
+int
+nvkm_nvdec_new_(struct nvkm_device *device, int index,
+ struct nvkm_nvdec **pnvdec)
+{
+ struct nvkm_nvdec *nvdec;
+
+ if (!(nvdec = *pnvdec = kzalloc(sizeof(*nvdec), GFP_KERNEL)))
+ return -ENOMEM;
+
+ return nvkm_engine_ctor(&nvkm_nvdec, device, index, true,
+ &nvdec->engine);
+};
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include "priv.h"
+
+int
+gp102_nvdec_new(struct nvkm_device *device, int index,
+ struct nvkm_nvdec **pnvdec)
+{
+ return nvkm_nvdec_new_(device, index, pnvdec);
+}
--- /dev/null
+#ifndef __NVKM_NVDEC_PRIV_H__
+#define __NVKM_NVDEC_PRIV_H__
+#include <engine/nvdec.h>
+
+int nvkm_nvdec_new_(struct nvkm_device *, int, struct nvkm_nvdec **);
+#endif
--- /dev/null
+nvkm-y += nvkm/engine/sec2/base.o
+nvkm-y += nvkm/engine/sec2/gp102.o
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+#include <core/msgqueue.h>
+#include <engine/falcon.h>
+
+static void *
+nvkm_sec2_dtor(struct nvkm_engine *engine)
+{
+ struct nvkm_sec2 *sec2 = nvkm_sec2(engine);
+ nvkm_msgqueue_del(&sec2->queue);
+ nvkm_falcon_del(&sec2->falcon);
+ return sec2;
+}
+
+static void
+nvkm_sec2_intr(struct nvkm_engine *engine)
+{
+ struct nvkm_sec2 *sec2 = nvkm_sec2(engine);
+ struct nvkm_subdev *subdev = &engine->subdev;
+ struct nvkm_device *device = subdev->device;
+ u32 disp = nvkm_rd32(device, 0x8701c);
+ u32 intr = nvkm_rd32(device, 0x87008) & disp & ~(disp >> 16);
+
+ if (intr & 0x00000040) {
+ schedule_work(&sec2->work);
+ nvkm_wr32(device, 0x87004, 0x00000040);
+ intr &= ~0x00000040;
+ }
+
+ if (intr) {
+ nvkm_error(subdev, "unhandled intr %08x\n", intr);
+ nvkm_wr32(device, 0x87004, intr);
+
+ }
+}
+
+static void
+nvkm_sec2_recv(struct work_struct *work)
+{
+ struct nvkm_sec2 *sec2 = container_of(work, typeof(*sec2), work);
+
+ if (!sec2->queue) {
+ nvkm_warn(&sec2->engine.subdev,
+ "recv function called while no firmware set!\n");
+ return;
+ }
+
+ nvkm_msgqueue_recv(sec2->queue);
+}
+
+
+static int
+nvkm_sec2_oneinit(struct nvkm_engine *engine)
+{
+ struct nvkm_sec2 *sec2 = nvkm_sec2(engine);
+ return nvkm_falcon_v1_new(&sec2->engine.subdev, "SEC2", 0x87000,
+ &sec2->falcon);
+}
+
+static int
+nvkm_sec2_fini(struct nvkm_engine *engine, bool suspend)
+{
+ struct nvkm_sec2 *sec2 = nvkm_sec2(engine);
+ flush_work(&sec2->work);
+ return 0;
+}
+
+static const struct nvkm_engine_func
+nvkm_sec2 = {
+ .dtor = nvkm_sec2_dtor,
+ .oneinit = nvkm_sec2_oneinit,
+ .fini = nvkm_sec2_fini,
+ .intr = nvkm_sec2_intr,
+};
+
+int
+nvkm_sec2_new_(struct nvkm_device *device, int index,
+ struct nvkm_sec2 **psec2)
+{
+ struct nvkm_sec2 *sec2;
+
+ if (!(sec2 = *psec2 = kzalloc(sizeof(*sec2), GFP_KERNEL)))
+ return -ENOMEM;
+ INIT_WORK(&sec2->work, nvkm_sec2_recv);
+
+ return nvkm_engine_ctor(&nvkm_sec2, device, index, true, &sec2->engine);
+};
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include "priv.h"
+
+int
+gp102_sec2_new(struct nvkm_device *device, int index,
+ struct nvkm_sec2 **psec2)
+{
+ return nvkm_sec2_new_(device, index, psec2);
+}
--- /dev/null
+#ifndef __NVKM_SEC2_PRIV_H__
+#define __NVKM_SEC2_PRIV_H__
+#include <engine/sec2.h>
+
+#define nvkm_sec2(p) container_of((p), struct nvkm_sec2, engine)
+
+int nvkm_sec2_new_(struct nvkm_device *, int, struct nvkm_sec2 **);
+
+#endif
nvkm-y += nvkm/falcon/base.o
nvkm-y += nvkm/falcon/v1.o
+nvkm-y += nvkm/falcon/msgqueue.o
+nvkm-y += nvkm/falcon/msgqueue_0137c63d.o
+nvkm-y += nvkm/falcon/msgqueue_0148cdec.o
nvkm_falcon_load_dmem(struct nvkm_falcon *falcon, void *data, u32 start,
u32 size, u8 port)
{
+ mutex_lock(&falcon->dmem_mutex);
+
falcon->func->load_dmem(falcon, data, start, size, port);
+
+ mutex_unlock(&falcon->dmem_mutex);
}
void
nvkm_falcon_read_dmem(struct nvkm_falcon *falcon, u32 start, u32 size, u8 port,
void *data)
{
+ mutex_lock(&falcon->dmem_mutex);
+
falcon->func->read_dmem(falcon, start, size, port, data);
+
+ mutex_unlock(&falcon->dmem_mutex);
}
void
void
nvkm_falcon_put(struct nvkm_falcon *falcon, const struct nvkm_subdev *user)
{
+ if (unlikely(!falcon))
+ return;
+
mutex_lock(&falcon->mutex);
if (falcon->user == user) {
nvkm_debug(falcon->user, "released %s falcon\n", falcon->name);
struct nvkm_subdev *subdev, const char *name, u32 addr,
struct nvkm_falcon *falcon)
{
+ u32 debug_reg;
u32 reg;
falcon->func = func;
falcon->name = name;
falcon->addr = addr;
mutex_init(&falcon->mutex);
+ mutex_init(&falcon->dmem_mutex);
reg = nvkm_falcon_rd32(falcon, 0x12c);
falcon->version = reg & 0xf;
falcon->code.limit = (reg & 0x1ff) << 8;
falcon->data.limit = (reg & 0x3fe00) >> 1;
- reg = nvkm_falcon_rd32(falcon, 0xc08);
- falcon->debug = (reg >> 20) & 0x1;
+ switch (subdev->index) {
+ case NVKM_ENGINE_GR:
+ debug_reg = 0x0;
+ break;
+ case NVKM_SUBDEV_PMU:
+ debug_reg = 0xc08;
+ break;
+ case NVKM_ENGINE_NVDEC:
+ debug_reg = 0xd00;
+ break;
+ case NVKM_ENGINE_SEC2:
+ debug_reg = 0x408;
+ falcon->has_emem = true;
+ break;
+ default:
+ nvkm_warn(subdev, "unsupported falcon %s!\n",
+ nvkm_subdev_name[subdev->index]);
+ debug_reg = 0;
+ break;
+ }
+
+ if (debug_reg) {
+ u32 val = nvkm_falcon_rd32(falcon, debug_reg);
+ falcon->debug = (val >> 20) & 0x1;
+ }
}
void
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "msgqueue.h"
+#include <engine/falcon.h>
+
+#include <subdev/secboot.h>
+
+
+#define HDR_SIZE sizeof(struct nvkm_msgqueue_hdr)
+#define QUEUE_ALIGNMENT 4
+/* max size of the messages we can receive */
+#define MSG_BUF_SIZE 128
+
+static int
+msg_queue_open(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_queue *queue)
+{
+ struct nvkm_falcon *falcon = priv->falcon;
+
+ mutex_lock(&queue->mutex);
+
+ queue->position = nvkm_falcon_rd32(falcon, queue->tail_reg);
+
+ return 0;
+}
+
+static void
+msg_queue_close(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_queue *queue,
+ bool commit)
+{
+ struct nvkm_falcon *falcon = priv->falcon;
+
+ if (commit)
+ nvkm_falcon_wr32(falcon, queue->tail_reg, queue->position);
+
+ mutex_unlock(&queue->mutex);
+}
+
+static bool
+msg_queue_empty(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_queue *queue)
+{
+ struct nvkm_falcon *falcon = priv->falcon;
+ u32 head, tail;
+
+ head = nvkm_falcon_rd32(falcon, queue->head_reg);
+ tail = nvkm_falcon_rd32(falcon, queue->tail_reg);
+
+ return head == tail;
+}
+
+static int
+msg_queue_pop(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_queue *queue,
+ void *data, u32 size)
+{
+ struct nvkm_falcon *falcon = priv->falcon;
+ const struct nvkm_subdev *subdev = priv->falcon->owner;
+ u32 head, tail, available;
+
+ head = nvkm_falcon_rd32(falcon, queue->head_reg);
+ /* has the buffer looped? */
+ if (head < queue->position)
+ queue->position = queue->offset;
+
+ tail = queue->position;
+
+ available = head - tail;
+
+ if (available == 0) {
+ nvkm_warn(subdev, "no message data available\n");
+ return 0;
+ }
+
+ if (size > available) {
+ nvkm_warn(subdev, "message data smaller than read request\n");
+ size = available;
+ }
+
+ nvkm_falcon_read_dmem(priv->falcon, tail, size, 0, data);
+ queue->position += ALIGN(size, QUEUE_ALIGNMENT);
+
+ return size;
+}
+
+static int
+msg_queue_read(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_queue *queue,
+ struct nvkm_msgqueue_hdr *hdr)
+{
+ const struct nvkm_subdev *subdev = priv->falcon->owner;
+ int err;
+
+ err = msg_queue_open(priv, queue);
+ if (err) {
+ nvkm_error(subdev, "fail to open queue %d\n", queue->index);
+ return err;
+ }
+
+ if (msg_queue_empty(priv, queue)) {
+ err = 0;
+ goto close;
+ }
+
+ err = msg_queue_pop(priv, queue, hdr, HDR_SIZE);
+ if (err >= 0 && err != HDR_SIZE)
+ err = -EINVAL;
+ if (err < 0) {
+ nvkm_error(subdev, "failed to read message header: %d\n", err);
+ goto close;
+ }
+
+ if (hdr->size > MSG_BUF_SIZE) {
+ nvkm_error(subdev, "message too big (%d bytes)\n", hdr->size);
+ err = -ENOSPC;
+ goto close;
+ }
+
+ if (hdr->size > HDR_SIZE) {
+ u32 read_size = hdr->size - HDR_SIZE;
+
+ err = msg_queue_pop(priv, queue, (hdr + 1), read_size);
+ if (err >= 0 && err != read_size)
+ err = -EINVAL;
+ if (err < 0) {
+ nvkm_error(subdev, "failed to read message: %d\n", err);
+ goto close;
+ }
+ }
+
+close:
+ msg_queue_close(priv, queue, (err >= 0));
+
+ return err;
+}
+
+static bool
+cmd_queue_has_room(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_queue *queue,
+ u32 size, bool *rewind)
+{
+ struct nvkm_falcon *falcon = priv->falcon;
+ u32 head, tail, free;
+
+ size = ALIGN(size, QUEUE_ALIGNMENT);
+
+ head = nvkm_falcon_rd32(falcon, queue->head_reg);
+ tail = nvkm_falcon_rd32(falcon, queue->tail_reg);
+
+ if (head >= tail) {
+ free = queue->offset + queue->size - head;
+ free -= HDR_SIZE;
+
+ if (size > free) {
+ *rewind = true;
+ head = queue->offset;
+ }
+ }
+
+ if (head < tail)
+ free = tail - head - 1;
+
+ return size <= free;
+}
+
+static int
+cmd_queue_push(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_queue *queue,
+ void *data, u32 size)
+{
+ nvkm_falcon_load_dmem(priv->falcon, data, queue->position, size, 0);
+ queue->position += ALIGN(size, QUEUE_ALIGNMENT);
+
+ return 0;
+}
+
+/* REWIND unit is always 0x00 */
+#define MSGQUEUE_UNIT_REWIND 0x00
+
+static void
+cmd_queue_rewind(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_queue *queue)
+{
+ const struct nvkm_subdev *subdev = priv->falcon->owner;
+ struct nvkm_msgqueue_hdr cmd;
+ int err;
+
+ cmd.unit_id = MSGQUEUE_UNIT_REWIND;
+ cmd.size = sizeof(cmd);
+ err = cmd_queue_push(priv, queue, &cmd, cmd.size);
+ if (err)
+ nvkm_error(subdev, "queue %d rewind failed\n", queue->index);
+ else
+ nvkm_error(subdev, "queue %d rewinded\n", queue->index);
+
+ queue->position = queue->offset;
+}
+
+static int
+cmd_queue_open(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_queue *queue,
+ u32 size)
+{
+ struct nvkm_falcon *falcon = priv->falcon;
+ const struct nvkm_subdev *subdev = priv->falcon->owner;
+ bool rewind = false;
+
+ mutex_lock(&queue->mutex);
+
+ if (!cmd_queue_has_room(priv, queue, size, &rewind)) {
+ nvkm_error(subdev, "queue full\n");
+ mutex_unlock(&queue->mutex);
+ return -EAGAIN;
+ }
+
+ queue->position = nvkm_falcon_rd32(falcon, queue->head_reg);
+
+ if (rewind)
+ cmd_queue_rewind(priv, queue);
+
+ return 0;
+}
+
+static void
+cmd_queue_close(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_queue *queue,
+ bool commit)
+{
+ struct nvkm_falcon *falcon = priv->falcon;
+
+ if (commit)
+ nvkm_falcon_wr32(falcon, queue->head_reg, queue->position);
+
+ mutex_unlock(&queue->mutex);
+}
+
+static int
+cmd_write(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_hdr *cmd,
+ struct nvkm_msgqueue_queue *queue)
+{
+ const struct nvkm_subdev *subdev = priv->falcon->owner;
+ static unsigned long timeout = ~0;
+ unsigned long end_jiffies = jiffies + msecs_to_jiffies(timeout);
+ int ret = -EAGAIN;
+ bool commit = true;
+
+ while (ret == -EAGAIN && time_before(jiffies, end_jiffies))
+ ret = cmd_queue_open(priv, queue, cmd->size);
+ if (ret) {
+ nvkm_error(subdev, "pmu_queue_open_write failed\n");
+ return ret;
+ }
+
+ ret = cmd_queue_push(priv, queue, cmd, cmd->size);
+ if (ret) {
+ nvkm_error(subdev, "pmu_queue_push failed\n");
+ commit = false;
+ }
+
+ cmd_queue_close(priv, queue, commit);
+
+ return ret;
+}
+
+static struct nvkm_msgqueue_seq *
+msgqueue_seq_acquire(struct nvkm_msgqueue *priv)
+{
+ const struct nvkm_subdev *subdev = priv->falcon->owner;
+ struct nvkm_msgqueue_seq *seq;
+ u32 index;
+
+ mutex_lock(&priv->seq_lock);
+
+ index = find_first_zero_bit(priv->seq_tbl, NVKM_MSGQUEUE_NUM_SEQUENCES);
+
+ if (index >= NVKM_MSGQUEUE_NUM_SEQUENCES) {
+ nvkm_error(subdev, "no free sequence available\n");
+ mutex_unlock(&priv->seq_lock);
+ return ERR_PTR(-EAGAIN);
+ }
+
+ set_bit(index, priv->seq_tbl);
+
+ mutex_unlock(&priv->seq_lock);
+
+ seq = &priv->seq[index];
+ seq->state = SEQ_STATE_PENDING;
+
+ return seq;
+}
+
+static void
+msgqueue_seq_release(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_seq *seq)
+{
+ /* no need to acquire seq_lock since clear_bit is atomic */
+ seq->state = SEQ_STATE_FREE;
+ seq->callback = NULL;
+ seq->completion = NULL;
+ clear_bit(seq->id, priv->seq_tbl);
+}
+
+/* specifies that we want to know the command status in the answer message */
+#define CMD_FLAGS_STATUS BIT(0)
+/* specifies that we want an interrupt when the answer message is queued */
+#define CMD_FLAGS_INTR BIT(1)
+
+int
+nvkm_msgqueue_post(struct nvkm_msgqueue *priv, enum msgqueue_msg_priority prio,
+ struct nvkm_msgqueue_hdr *cmd, nvkm_msgqueue_callback cb,
+ struct completion *completion, bool wait_init)
+{
+ struct nvkm_msgqueue_seq *seq;
+ struct nvkm_msgqueue_queue *queue;
+ int ret;
+
+ if (wait_init && !wait_for_completion_timeout(&priv->init_done,
+ msecs_to_jiffies(1000)))
+ return -ETIMEDOUT;
+
+ queue = priv->func->cmd_queue(priv, prio);
+ if (IS_ERR(queue))
+ return PTR_ERR(queue);
+
+ seq = msgqueue_seq_acquire(priv);
+ if (IS_ERR(seq))
+ return PTR_ERR(seq);
+
+ cmd->seq_id = seq->id;
+ cmd->ctrl_flags = CMD_FLAGS_STATUS | CMD_FLAGS_INTR;
+
+ seq->callback = cb;
+ seq->state = SEQ_STATE_USED;
+ seq->completion = completion;
+
+ ret = cmd_write(priv, cmd, queue);
+ if (ret) {
+ seq->state = SEQ_STATE_PENDING;
+ msgqueue_seq_release(priv, seq);
+ }
+
+ return ret;
+}
+
+static int
+msgqueue_msg_handle(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_hdr *hdr)
+{
+ const struct nvkm_subdev *subdev = priv->falcon->owner;
+ struct nvkm_msgqueue_seq *seq;
+
+ seq = &priv->seq[hdr->seq_id];
+ if (seq->state != SEQ_STATE_USED && seq->state != SEQ_STATE_CANCELLED) {
+ nvkm_error(subdev, "msg for unknown sequence %d", seq->id);
+ return -EINVAL;
+ }
+
+ if (seq->state == SEQ_STATE_USED) {
+ if (seq->callback)
+ seq->callback(priv, hdr);
+ }
+
+ if (seq->completion)
+ complete(seq->completion);
+
+ msgqueue_seq_release(priv, seq);
+
+ return 0;
+}
+
+static int
+msgqueue_handle_init_msg(struct nvkm_msgqueue *priv,
+ struct nvkm_msgqueue_hdr *hdr)
+{
+ struct nvkm_falcon *falcon = priv->falcon;
+ const struct nvkm_subdev *subdev = falcon->owner;
+ u32 tail;
+ u32 tail_reg;
+ int ret;
+
+ /*
+ * Of course the message queue registers vary depending on the falcon
+ * used...
+ */
+ switch (falcon->owner->index) {
+ case NVKM_SUBDEV_PMU:
+ tail_reg = 0x4cc;
+ break;
+ case NVKM_ENGINE_SEC2:
+ tail_reg = 0xa34;
+ break;
+ default:
+ nvkm_error(subdev, "falcon %s unsupported for msgqueue!\n",
+ nvkm_subdev_name[falcon->owner->index]);
+ return -EINVAL;
+ }
+
+ /*
+ * Read the message - queues are not initialized yet so we cannot rely
+ * on msg_queue_read()
+ */
+ tail = nvkm_falcon_rd32(falcon, tail_reg);
+ nvkm_falcon_read_dmem(falcon, tail, HDR_SIZE, 0, hdr);
+
+ if (hdr->size > MSG_BUF_SIZE) {
+ nvkm_error(subdev, "message too big (%d bytes)\n", hdr->size);
+ return -ENOSPC;
+ }
+
+ nvkm_falcon_read_dmem(falcon, tail + HDR_SIZE, hdr->size - HDR_SIZE, 0,
+ (hdr + 1));
+
+ tail += ALIGN(hdr->size, QUEUE_ALIGNMENT);
+ nvkm_falcon_wr32(falcon, tail_reg, tail);
+
+ ret = priv->func->init_func->init_callback(priv, hdr);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+void
+nvkm_msgqueue_process_msgs(struct nvkm_msgqueue *priv,
+ struct nvkm_msgqueue_queue *queue)
+{
+ /*
+ * We are invoked from a worker thread, so normally we have plenty of
+ * stack space to work with.
+ */
+ u8 msg_buffer[MSG_BUF_SIZE];
+ struct nvkm_msgqueue_hdr *hdr = (void *)msg_buffer;
+ int ret;
+
+ /* the first message we receive must be the init message */
+ if ((!priv->init_msg_received)) {
+ ret = msgqueue_handle_init_msg(priv, hdr);
+ if (!ret)
+ priv->init_msg_received = true;
+ } else {
+ while (msg_queue_read(priv, queue, hdr) > 0)
+ msgqueue_msg_handle(priv, hdr);
+ }
+}
+
+void
+nvkm_msgqueue_write_cmdline(struct nvkm_msgqueue *queue, void *buf)
+{
+ if (!queue || !queue->func || !queue->func->init_func)
+ return;
+
+ queue->func->init_func->gen_cmdline(queue, buf);
+}
+
+int
+nvkm_msgqueue_acr_boot_falcon(struct nvkm_msgqueue *queue, enum nvkm_secboot_falcon falcon)
+{
+ if (!queue || !queue->func->acr_func || !queue->func->acr_func->boot_falcon)
+ return -ENODEV;
+
+ return queue->func->acr_func->boot_falcon(queue, falcon);
+}
+
+int
+nvkm_msgqueue_new(u32 version, struct nvkm_falcon *falcon, struct nvkm_msgqueue **queue)
+{
+ const struct nvkm_subdev *subdev = falcon->owner;
+ int ret = -EINVAL;
+
+ switch (version) {
+ case 0x0137c63d:
+ ret = msgqueue_0137c63d_new(falcon, queue);
+ break;
+ case 0x0148cdec:
+ ret = msgqueue_0148cdec_new(falcon, queue);
+ break;
+ default:
+ nvkm_error(subdev, "unhandled firmware version 0x%08x\n",
+ version);
+ break;
+ }
+
+ if (ret == 0) {
+ nvkm_debug(subdev, "firmware version: 0x%08x\n", version);
+ (*queue)->fw_version = version;
+ }
+
+ return ret;
+}
+
+void
+nvkm_msgqueue_del(struct nvkm_msgqueue **queue)
+{
+ if (*queue) {
+ (*queue)->func->dtor(*queue);
+ *queue = NULL;
+ }
+}
+
+void
+nvkm_msgqueue_recv(struct nvkm_msgqueue *queue)
+{
+ if (!queue->func || !queue->func->recv) {
+ const struct nvkm_subdev *subdev = queue->falcon->owner;
+
+ nvkm_warn(subdev, "missing msgqueue recv function\n");
+ return;
+ }
+
+ queue->func->recv(queue);
+}
+
+int
+nvkm_msgqueue_reinit(struct nvkm_msgqueue *queue)
+{
+ /* firmware not set yet... */
+ if (!queue)
+ return 0;
+
+ queue->init_msg_received = false;
+ reinit_completion(&queue->init_done);
+
+ return 0;
+}
+
+void
+nvkm_msgqueue_ctor(const struct nvkm_msgqueue_func *func,
+ struct nvkm_falcon *falcon,
+ struct nvkm_msgqueue *queue)
+{
+ int i;
+
+ queue->func = func;
+ queue->falcon = falcon;
+ mutex_init(&queue->seq_lock);
+ for (i = 0; i < NVKM_MSGQUEUE_NUM_SEQUENCES; i++)
+ queue->seq[i].id = i;
+
+ init_completion(&queue->init_done);
+
+
+}
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __NVKM_CORE_FALCON_MSGQUEUE_H
+#define __NVKM_CORE_FALCON_MSGQUEUE_H
+
+#include <core/msgqueue.h>
+
+/*
+ * The struct nvkm_msgqueue (named so for lack of better candidate) manages
+ * a firmware (typically, NVIDIA signed firmware) running under a given falcon.
+ *
+ * Such firmwares expect to receive commands (through one or several command
+ * queues) and will reply to such command by sending messages (using one
+ * message queue).
+ *
+ * Each firmware can support one or several units - ACR for managing secure
+ * falcons, PMU for power management, etc. A unit can be seen as a class to
+ * which command can be sent.
+ *
+ * One usage example would be to send a command to the SEC falcon to ask it to
+ * reset a secure falcon. The SEC falcon will receive the command, process it,
+ * and send a message to signal success or failure. Only when the corresponding
+ * message is received can the requester assume the request has been processed.
+ *
+ * Since we expect many variations between the firmwares NVIDIA will release
+ * across GPU generations, this library is built in a very modular way. Message
+ * formats and queues details (such as number of usage) are left to
+ * specializations of struct nvkm_msgqueue, while the functions in msgqueue.c
+ * take care of posting commands and processing messages in a fashion that is
+ * universal.
+ *
+ */
+
+enum msgqueue_msg_priority {
+ MSGQUEUE_MSG_PRIORITY_HIGH,
+ MSGQUEUE_MSG_PRIORITY_LOW,
+};
+
+/**
+ * struct nvkm_msgqueue_hdr - header for all commands/messages
+ * @unit_id: id of firmware using receiving the command/sending the message
+ * @size: total size of command/message
+ * @ctrl_flags: type of command/message
+ * @seq_id: used to match a message from its corresponding command
+ */
+struct nvkm_msgqueue_hdr {
+ u8 unit_id;
+ u8 size;
+ u8 ctrl_flags;
+ u8 seq_id;
+};
+
+/**
+ * struct nvkm_msgqueue_msg - base message.
+ *
+ * This is just a header and a message (or command) type. Useful when
+ * building command-specific structures.
+ */
+struct nvkm_msgqueue_msg {
+ struct nvkm_msgqueue_hdr hdr;
+ u8 msg_type;
+};
+
+struct nvkm_msgqueue;
+typedef void
+(*nvkm_msgqueue_callback)(struct nvkm_msgqueue *, struct nvkm_msgqueue_hdr *);
+
+/**
+ * struct nvkm_msgqueue_init_func - msgqueue functions related to initialization
+ *
+ * @gen_cmdline: build the commandline into a pre-allocated buffer
+ * @init_callback: called to process the init message
+ */
+struct nvkm_msgqueue_init_func {
+ void (*gen_cmdline)(struct nvkm_msgqueue *, void *);
+ int (*init_callback)(struct nvkm_msgqueue *, struct nvkm_msgqueue_hdr *);
+};
+
+/**
+ * struct nvkm_msgqueue_acr_func - msgqueue functions related to ACR
+ *
+ * @boot_falcon: build and send the command to reset a given falcon
+ */
+struct nvkm_msgqueue_acr_func {
+ int (*boot_falcon)(struct nvkm_msgqueue *, enum nvkm_secboot_falcon);
+};
+
+struct nvkm_msgqueue_func {
+ const struct nvkm_msgqueue_init_func *init_func;
+ const struct nvkm_msgqueue_acr_func *acr_func;
+ void (*dtor)(struct nvkm_msgqueue *);
+ struct nvkm_msgqueue_queue *(*cmd_queue)(struct nvkm_msgqueue *,
+ enum msgqueue_msg_priority);
+ void (*recv)(struct nvkm_msgqueue *queue);
+};
+
+/**
+ * struct nvkm_msgqueue_queue - information about a command or message queue
+ *
+ * The number of queues is firmware-dependent. All queues must have their
+ * information filled by the init message handler.
+ *
+ * @mutex_lock: to be acquired when the queue is being used
+ * @index: physical queue index
+ * @offset: DMEM offset where this queue begins
+ * @size: size allocated to this queue in DMEM (in bytes)
+ * @position: current write position
+ * @head_reg: address of the HEAD register for this queue
+ * @tail_reg: address of the TAIL register for this queue
+ */
+struct nvkm_msgqueue_queue {
+ struct mutex mutex;
+ u32 index;
+ u32 offset;
+ u32 size;
+ u32 position;
+
+ u32 head_reg;
+ u32 tail_reg;
+};
+
+/**
+ * struct nvkm_msgqueue_seq - keep track of ongoing commands
+ *
+ * Every time a command is sent, a sequence is assigned to it so the
+ * corresponding message can be matched. Upon receiving the message, a callback
+ * can be called and/or a completion signaled.
+ *
+ * @id: sequence ID
+ * @state: current state
+ * @callback: callback to call upon receiving matching message
+ * @completion: completion to signal after callback is called
+ */
+struct nvkm_msgqueue_seq {
+ u16 id;
+ enum {
+ SEQ_STATE_FREE = 0,
+ SEQ_STATE_PENDING,
+ SEQ_STATE_USED,
+ SEQ_STATE_CANCELLED
+ } state;
+ nvkm_msgqueue_callback callback;
+ struct completion *completion;
+};
+
+/*
+ * We can have an arbitrary number of sequences, but realistically we will
+ * probably not use that much simultaneously.
+ */
+#define NVKM_MSGQUEUE_NUM_SEQUENCES 16
+
+/**
+ * struct nvkm_msgqueue - manage a command/message based FW on a falcon
+ *
+ * @falcon: falcon to be managed
+ * @func: implementation of the firmware to use
+ * @init_msg_received: whether the init message has already been received
+ * @init_done: whether all init is complete and commands can be processed
+ * @seq_lock: protects seq and seq_tbl
+ * @seq: sequences to match commands and messages
+ * @seq_tbl: bitmap of sequences currently in use
+ */
+struct nvkm_msgqueue {
+ struct nvkm_falcon *falcon;
+ const struct nvkm_msgqueue_func *func;
+ u32 fw_version;
+ bool init_msg_received;
+ struct completion init_done;
+
+ struct mutex seq_lock;
+ struct nvkm_msgqueue_seq seq[NVKM_MSGQUEUE_NUM_SEQUENCES];
+ unsigned long seq_tbl[BITS_TO_LONGS(NVKM_MSGQUEUE_NUM_SEQUENCES)];
+};
+
+void nvkm_msgqueue_ctor(const struct nvkm_msgqueue_func *, struct nvkm_falcon *,
+ struct nvkm_msgqueue *);
+int nvkm_msgqueue_post(struct nvkm_msgqueue *, enum msgqueue_msg_priority,
+ struct nvkm_msgqueue_hdr *, nvkm_msgqueue_callback,
+ struct completion *, bool);
+void nvkm_msgqueue_process_msgs(struct nvkm_msgqueue *,
+ struct nvkm_msgqueue_queue *);
+
+int msgqueue_0137c63d_new(struct nvkm_falcon *, struct nvkm_msgqueue **);
+int msgqueue_0148cdec_new(struct nvkm_falcon *, struct nvkm_msgqueue **);
+
+#endif
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#include "msgqueue.h"
+#include <engine/falcon.h>
+#include <subdev/secboot.h>
+
+/* Queues identifiers */
+enum {
+ /* High Priority Command Queue for Host -> PMU communication */
+ MSGQUEUE_0137C63D_COMMAND_QUEUE_HPQ = 0,
+ /* Low Priority Command Queue for Host -> PMU communication */
+ MSGQUEUE_0137C63D_COMMAND_QUEUE_LPQ = 1,
+ /* Message queue for PMU -> Host communication */
+ MSGQUEUE_0137C63D_MESSAGE_QUEUE = 4,
+ MSGQUEUE_0137C63D_NUM_QUEUES = 5,
+};
+
+struct msgqueue_0137c63d {
+ struct nvkm_msgqueue base;
+
+ struct nvkm_msgqueue_queue queue[MSGQUEUE_0137C63D_NUM_QUEUES];
+};
+#define msgqueue_0137c63d(q) \
+ container_of(q, struct msgqueue_0137c63d, base)
+
+static struct nvkm_msgqueue_queue *
+msgqueue_0137c63d_cmd_queue(struct nvkm_msgqueue *queue,
+ enum msgqueue_msg_priority priority)
+{
+ struct msgqueue_0137c63d *priv = msgqueue_0137c63d(queue);
+ const struct nvkm_subdev *subdev = priv->base.falcon->owner;
+
+ switch (priority) {
+ case MSGQUEUE_MSG_PRIORITY_HIGH:
+ return &priv->queue[MSGQUEUE_0137C63D_COMMAND_QUEUE_HPQ];
+ case MSGQUEUE_MSG_PRIORITY_LOW:
+ return &priv->queue[MSGQUEUE_0137C63D_COMMAND_QUEUE_LPQ];
+ default:
+ nvkm_error(subdev, "invalid command queue!\n");
+ return ERR_PTR(-EINVAL);
+ }
+}
+
+static void
+msgqueue_0137c63d_process_msgs(struct nvkm_msgqueue *queue)
+{
+ struct msgqueue_0137c63d *priv = msgqueue_0137c63d(queue);
+ struct nvkm_msgqueue_queue *q_queue =
+ &priv->queue[MSGQUEUE_0137C63D_MESSAGE_QUEUE];
+
+ nvkm_msgqueue_process_msgs(&priv->base, q_queue);
+}
+
+/* Init unit */
+#define MSGQUEUE_0137C63D_UNIT_INIT 0x07
+
+enum {
+ INIT_MSG_INIT = 0x0,
+};
+
+static void
+init_gen_cmdline(struct nvkm_msgqueue *queue, void *buf)
+{
+ struct {
+ u32 reserved;
+ u32 freq_hz;
+ u32 trace_size;
+ u32 trace_dma_base;
+ u16 trace_dma_base1;
+ u8 trace_dma_offset;
+ u32 trace_dma_idx;
+ bool secure_mode;
+ bool raise_priv_sec;
+ struct {
+ u32 dma_base;
+ u16 dma_base1;
+ u8 dma_offset;
+ u16 fb_size;
+ u8 dma_idx;
+ } gc6_ctx;
+ u8 pad;
+ } *args = buf;
+
+ args->secure_mode = 1;
+}
+
+/* forward declaration */
+static int acr_init_wpr(struct nvkm_msgqueue *queue);
+
+static int
+init_callback(struct nvkm_msgqueue *_queue, struct nvkm_msgqueue_hdr *hdr)
+{
+ struct msgqueue_0137c63d *priv = msgqueue_0137c63d(_queue);
+ struct {
+ struct nvkm_msgqueue_msg base;
+
+ u8 pad;
+ u16 os_debug_entry_point;
+
+ struct {
+ u16 size;
+ u16 offset;
+ u8 index;
+ u8 pad;
+ } queue_info[MSGQUEUE_0137C63D_NUM_QUEUES];
+
+ u16 sw_managed_area_offset;
+ u16 sw_managed_area_size;
+ } *init = (void *)hdr;
+ const struct nvkm_subdev *subdev = _queue->falcon->owner;
+ int i;
+
+ if (init->base.hdr.unit_id != MSGQUEUE_0137C63D_UNIT_INIT) {
+ nvkm_error(subdev, "expected message from init unit\n");
+ return -EINVAL;
+ }
+
+ if (init->base.msg_type != INIT_MSG_INIT) {
+ nvkm_error(subdev, "expected PMU init msg\n");
+ return -EINVAL;
+ }
+
+ for (i = 0; i < MSGQUEUE_0137C63D_NUM_QUEUES; i++) {
+ struct nvkm_msgqueue_queue *queue = &priv->queue[i];
+
+ mutex_init(&queue->mutex);
+
+ queue->index = init->queue_info[i].index;
+ queue->offset = init->queue_info[i].offset;
+ queue->size = init->queue_info[i].size;
+
+ if (i != MSGQUEUE_0137C63D_MESSAGE_QUEUE) {
+ queue->head_reg = 0x4a0 + (queue->index * 4);
+ queue->tail_reg = 0x4b0 + (queue->index * 4);
+ } else {
+ queue->head_reg = 0x4c8;
+ queue->tail_reg = 0x4cc;
+ }
+
+ nvkm_debug(subdev,
+ "queue %d: index %d, offset 0x%08x, size 0x%08x\n",
+ i, queue->index, queue->offset, queue->size);
+ }
+
+ /* Complete initialization by initializing WPR region */
+ return acr_init_wpr(&priv->base);
+}
+
+static const struct nvkm_msgqueue_init_func
+msgqueue_0137c63d_init_func = {
+ .gen_cmdline = init_gen_cmdline,
+ .init_callback = init_callback,
+};
+
+
+
+/* ACR unit */
+#define MSGQUEUE_0137C63D_UNIT_ACR 0x0a
+
+enum {
+ ACR_CMD_INIT_WPR_REGION = 0x00,
+ ACR_CMD_BOOTSTRAP_FALCON = 0x01,
+};
+
+static void
+acr_init_wpr_callback(struct nvkm_msgqueue *queue,
+ struct nvkm_msgqueue_hdr *hdr)
+{
+ struct {
+ struct nvkm_msgqueue_msg base;
+ u32 error_code;
+ } *msg = (void *)hdr;
+ const struct nvkm_subdev *subdev = queue->falcon->owner;
+
+ if (msg->error_code) {
+ nvkm_error(subdev, "ACR WPR init failure: %d\n",
+ msg->error_code);
+ return;
+ }
+
+ nvkm_debug(subdev, "ACR WPR init complete\n");
+ complete_all(&queue->init_done);
+}
+
+static int
+acr_init_wpr(struct nvkm_msgqueue *queue)
+{
+ /*
+ * region_id: region ID in WPR region
+ * wpr_offset: offset in WPR region
+ */
+ struct {
+ struct nvkm_msgqueue_hdr hdr;
+ u8 cmd_type;
+ u32 region_id;
+ u32 wpr_offset;
+ } cmd;
+ memset(&cmd, 0, sizeof(cmd));
+
+ cmd.hdr.unit_id = MSGQUEUE_0137C63D_UNIT_ACR;
+ cmd.hdr.size = sizeof(cmd);
+ cmd.cmd_type = ACR_CMD_INIT_WPR_REGION;
+ cmd.region_id = 0x01;
+ cmd.wpr_offset = 0x00;
+
+ nvkm_msgqueue_post(queue, MSGQUEUE_MSG_PRIORITY_HIGH, &cmd.hdr,
+ acr_init_wpr_callback, NULL, false);
+
+ return 0;
+}
+
+
+static void
+acr_boot_falcon_callback(struct nvkm_msgqueue *priv,
+ struct nvkm_msgqueue_hdr *hdr)
+{
+ struct acr_bootstrap_falcon_msg {
+ struct nvkm_msgqueue_msg base;
+
+ u32 falcon_id;
+ } *msg = (void *)hdr;
+ const struct nvkm_subdev *subdev = priv->falcon->owner;
+ u32 falcon_id = msg->falcon_id;
+
+ if (falcon_id >= NVKM_SECBOOT_FALCON_END) {
+ nvkm_error(subdev, "in bootstrap falcon callback:\n");
+ nvkm_error(subdev, "invalid falcon ID 0x%x\n", falcon_id);
+ return;
+ }
+ nvkm_debug(subdev, "%s booted\n", nvkm_secboot_falcon_name[falcon_id]);
+}
+
+enum {
+ ACR_CMD_BOOTSTRAP_FALCON_FLAGS_RESET_YES = 0,
+ ACR_CMD_BOOTSTRAP_FALCON_FLAGS_RESET_NO = 1,
+};
+
+static int
+acr_boot_falcon(struct nvkm_msgqueue *priv, enum nvkm_secboot_falcon falcon)
+{
+ DECLARE_COMPLETION_ONSTACK(completed);
+ /*
+ * flags - Flag specifying RESET or no RESET.
+ * falcon id - Falcon id specifying falcon to bootstrap.
+ */
+ struct {
+ struct nvkm_msgqueue_hdr hdr;
+ u8 cmd_type;
+ u32 flags;
+ u32 falcon_id;
+ } cmd;
+
+ memset(&cmd, 0, sizeof(cmd));
+
+ cmd.hdr.unit_id = MSGQUEUE_0137C63D_UNIT_ACR;
+ cmd.hdr.size = sizeof(cmd);
+ cmd.cmd_type = ACR_CMD_BOOTSTRAP_FALCON;
+ cmd.flags = ACR_CMD_BOOTSTRAP_FALCON_FLAGS_RESET_YES;
+ cmd.falcon_id = falcon;
+ nvkm_msgqueue_post(priv, MSGQUEUE_MSG_PRIORITY_HIGH, &cmd.hdr,
+ acr_boot_falcon_callback, &completed, true);
+
+ if (!wait_for_completion_timeout(&completed, msecs_to_jiffies(1000)))
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+static const struct nvkm_msgqueue_acr_func
+msgqueue_0137c63d_acr_func = {
+ .boot_falcon = acr_boot_falcon,
+};
+
+static void
+msgqueue_0137c63d_dtor(struct nvkm_msgqueue *queue)
+{
+ kfree(msgqueue_0137c63d(queue));
+}
+
+static const struct nvkm_msgqueue_func
+msgqueue_0137c63d_func = {
+ .init_func = &msgqueue_0137c63d_init_func,
+ .acr_func = &msgqueue_0137c63d_acr_func,
+ .cmd_queue = msgqueue_0137c63d_cmd_queue,
+ .recv = msgqueue_0137c63d_process_msgs,
+ .dtor = msgqueue_0137c63d_dtor,
+};
+
+int
+msgqueue_0137c63d_new(struct nvkm_falcon *falcon, struct nvkm_msgqueue **queue)
+{
+ struct msgqueue_0137c63d *ret;
+
+ ret = kzalloc(sizeof(*ret), GFP_KERNEL);
+ if (!ret)
+ return -ENOMEM;
+
+ *queue = &ret->base;
+
+ nvkm_msgqueue_ctor(&msgqueue_0137c63d_func, falcon, &ret->base);
+
+ return 0;
+}
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "msgqueue.h"
+#include <engine/falcon.h>
+#include <subdev/secboot.h>
+
+/*
+ * This firmware runs on the SEC falcon. It only has one command and one
+ * message queue, and uses a different command line and init message.
+ */
+
+enum {
+ MSGQUEUE_0148CDEC_COMMAND_QUEUE = 0,
+ MSGQUEUE_0148CDEC_MESSAGE_QUEUE = 1,
+ MSGQUEUE_0148CDEC_NUM_QUEUES,
+};
+
+struct msgqueue_0148cdec {
+ struct nvkm_msgqueue base;
+
+ struct nvkm_msgqueue_queue queue[MSGQUEUE_0148CDEC_NUM_QUEUES];
+};
+#define msgqueue_0148cdec(q) \
+ container_of(q, struct msgqueue_0148cdec, base)
+
+static struct nvkm_msgqueue_queue *
+msgqueue_0148cdec_cmd_queue(struct nvkm_msgqueue *queue,
+ enum msgqueue_msg_priority priority)
+{
+ struct msgqueue_0148cdec *priv = msgqueue_0148cdec(queue);
+
+ return &priv->queue[MSGQUEUE_0148CDEC_COMMAND_QUEUE];
+}
+
+static void
+msgqueue_0148cdec_process_msgs(struct nvkm_msgqueue *queue)
+{
+ struct msgqueue_0148cdec *priv = msgqueue_0148cdec(queue);
+ struct nvkm_msgqueue_queue *q_queue =
+ &priv->queue[MSGQUEUE_0148CDEC_MESSAGE_QUEUE];
+
+ nvkm_msgqueue_process_msgs(&priv->base, q_queue);
+}
+
+
+/* Init unit */
+#define MSGQUEUE_0148CDEC_UNIT_INIT 0x01
+
+enum {
+ INIT_MSG_INIT = 0x0,
+};
+
+static void
+init_gen_cmdline(struct nvkm_msgqueue *queue, void *buf)
+{
+ struct {
+ u32 freq_hz;
+ u32 falc_trace_size;
+ u32 falc_trace_dma_base;
+ u32 falc_trace_dma_idx;
+ bool secure_mode;
+ } *args = buf;
+
+ args->secure_mode = false;
+}
+
+static int
+init_callback(struct nvkm_msgqueue *_queue, struct nvkm_msgqueue_hdr *hdr)
+{
+ struct msgqueue_0148cdec *priv = msgqueue_0148cdec(_queue);
+ struct {
+ struct nvkm_msgqueue_msg base;
+
+ u8 num_queues;
+ u16 os_debug_entry_point;
+
+ struct {
+ u32 offset;
+ u16 size;
+ u8 index;
+ u8 id;
+ } queue_info[MSGQUEUE_0148CDEC_NUM_QUEUES];
+
+ u16 sw_managed_area_offset;
+ u16 sw_managed_area_size;
+ } *init = (void *)hdr;
+ const struct nvkm_subdev *subdev = _queue->falcon->owner;
+ int i;
+
+ if (init->base.hdr.unit_id != MSGQUEUE_0148CDEC_UNIT_INIT) {
+ nvkm_error(subdev, "expected message from init unit\n");
+ return -EINVAL;
+ }
+
+ if (init->base.msg_type != INIT_MSG_INIT) {
+ nvkm_error(subdev, "expected SEC init msg\n");
+ return -EINVAL;
+ }
+
+ for (i = 0; i < MSGQUEUE_0148CDEC_NUM_QUEUES; i++) {
+ u8 id = init->queue_info[i].id;
+ struct nvkm_msgqueue_queue *queue = &priv->queue[id];
+
+ mutex_init(&queue->mutex);
+
+ queue->index = init->queue_info[i].index;
+ queue->offset = init->queue_info[i].offset;
+ queue->size = init->queue_info[i].size;
+
+ if (id == MSGQUEUE_0148CDEC_MESSAGE_QUEUE) {
+ queue->head_reg = 0xa30 + (queue->index * 8);
+ queue->tail_reg = 0xa34 + (queue->index * 8);
+ } else {
+ queue->head_reg = 0xa00 + (queue->index * 8);
+ queue->tail_reg = 0xa04 + (queue->index * 8);
+ }
+
+ nvkm_debug(subdev,
+ "queue %d: index %d, offset 0x%08x, size 0x%08x\n",
+ id, queue->index, queue->offset, queue->size);
+ }
+
+ complete_all(&_queue->init_done);
+
+ return 0;
+}
+
+static const struct nvkm_msgqueue_init_func
+msgqueue_0148cdec_init_func = {
+ .gen_cmdline = init_gen_cmdline,
+ .init_callback = init_callback,
+};
+
+
+
+/* ACR unit */
+#define MSGQUEUE_0148CDEC_UNIT_ACR 0x08
+
+enum {
+ ACR_CMD_BOOTSTRAP_FALCON = 0x00,
+};
+
+static void
+acr_boot_falcon_callback(struct nvkm_msgqueue *priv,
+ struct nvkm_msgqueue_hdr *hdr)
+{
+ struct acr_bootstrap_falcon_msg {
+ struct nvkm_msgqueue_msg base;
+
+ u32 error_code;
+ u32 falcon_id;
+ } *msg = (void *)hdr;
+ const struct nvkm_subdev *subdev = priv->falcon->owner;
+ u32 falcon_id = msg->falcon_id;
+
+ if (msg->error_code) {
+ nvkm_error(subdev, "in bootstrap falcon callback:\n");
+ nvkm_error(subdev, "expected error code 0x%x\n",
+ msg->error_code);
+ return;
+ }
+
+ if (falcon_id >= NVKM_SECBOOT_FALCON_END) {
+ nvkm_error(subdev, "in bootstrap falcon callback:\n");
+ nvkm_error(subdev, "invalid falcon ID 0x%x\n", falcon_id);
+ return;
+ }
+
+ nvkm_debug(subdev, "%s booted\n", nvkm_secboot_falcon_name[falcon_id]);
+}
+
+enum {
+ ACR_CMD_BOOTSTRAP_FALCON_FLAGS_RESET_YES = 0,
+ ACR_CMD_BOOTSTRAP_FALCON_FLAGS_RESET_NO = 1,
+};
+
+static int
+acr_boot_falcon(struct nvkm_msgqueue *priv, enum nvkm_secboot_falcon falcon)
+{
+ DECLARE_COMPLETION_ONSTACK(completed);
+ /*
+ * flags - Flag specifying RESET or no RESET.
+ * falcon id - Falcon id specifying falcon to bootstrap.
+ */
+ struct {
+ struct nvkm_msgqueue_hdr hdr;
+ u8 cmd_type;
+ u32 flags;
+ u32 falcon_id;
+ } cmd;
+
+ memset(&cmd, 0, sizeof(cmd));
+
+ cmd.hdr.unit_id = MSGQUEUE_0148CDEC_UNIT_ACR;
+ cmd.hdr.size = sizeof(cmd);
+ cmd.cmd_type = ACR_CMD_BOOTSTRAP_FALCON;
+ cmd.flags = ACR_CMD_BOOTSTRAP_FALCON_FLAGS_RESET_YES;
+ cmd.falcon_id = falcon;
+ nvkm_msgqueue_post(priv, MSGQUEUE_MSG_PRIORITY_HIGH, &cmd.hdr,
+ acr_boot_falcon_callback, &completed, true);
+
+ if (!wait_for_completion_timeout(&completed, msecs_to_jiffies(1000)))
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+const struct nvkm_msgqueue_acr_func
+msgqueue_0148cdec_acr_func = {
+ .boot_falcon = acr_boot_falcon,
+};
+
+static void
+msgqueue_0148cdec_dtor(struct nvkm_msgqueue *queue)
+{
+ kfree(msgqueue_0148cdec(queue));
+}
+
+const struct nvkm_msgqueue_func
+msgqueue_0148cdec_func = {
+ .init_func = &msgqueue_0148cdec_init_func,
+ .acr_func = &msgqueue_0148cdec_acr_func,
+ .cmd_queue = msgqueue_0148cdec_cmd_queue,
+ .recv = msgqueue_0148cdec_process_msgs,
+ .dtor = msgqueue_0148cdec_dtor,
+};
+
+int
+msgqueue_0148cdec_new(struct nvkm_falcon *falcon, struct nvkm_msgqueue **queue)
+{
+ struct msgqueue_0148cdec *ret;
+
+ ret = kzalloc(sizeof(*ret), GFP_KERNEL);
+ if (!ret)
+ return -ENOMEM;
+
+ *queue = &ret->base;
+
+ nvkm_msgqueue_ctor(&msgqueue_0148cdec_func, falcon, &ret->base);
+
+ return 0;
+}
for (i = 0; i < size / 4; i++) {
/* write new tag every 256B */
if ((i & 0x3f) == 0)
- nvkm_falcon_wr32(falcon, 0x188, tag++);
- nvkm_falcon_wr32(falcon, 0x184, ((u32 *)data)[i]);
+ nvkm_falcon_wr32(falcon, 0x188 + (port * 16), tag++);
+ nvkm_falcon_wr32(falcon, 0x184 + (port * 16), ((u32 *)data)[i]);
}
/*
/* write new tag every 256B */
if ((i & 0x3f) == 0)
- nvkm_falcon_wr32(falcon, 0x188, tag++);
- nvkm_falcon_wr32(falcon, 0x184, extra & (BIT(rem * 8) - 1));
+ nvkm_falcon_wr32(falcon, 0x188 + (port * 16), tag++);
+ nvkm_falcon_wr32(falcon, 0x184 + (port * 16),
+ extra & (BIT(rem * 8) - 1));
++i;
}
/* code must be padded to 0x40 words */
for (; i & 0x3f; i++)
- nvkm_falcon_wr32(falcon, 0x184, 0);
+ nvkm_falcon_wr32(falcon, 0x184 + (port * 16), 0);
}
+static void
+nvkm_falcon_v1_load_emem(struct nvkm_falcon *falcon, void *data, u32 start,
+ u32 size, u8 port)
+{
+ u8 rem = size % 4;
+ int i;
+
+ size -= rem;
+
+ nvkm_falcon_wr32(falcon, 0xac0 + (port * 8), start | (0x1 << 24));
+ for (i = 0; i < size / 4; i++)
+ nvkm_falcon_wr32(falcon, 0xac4 + (port * 8), ((u32 *)data)[i]);
+
+ /*
+ * If size is not a multiple of 4, mask the last word to ensure garbage
+ * does not get written
+ */
+ if (rem) {
+ u32 extra = ((u32 *)data)[i];
+
+ nvkm_falcon_wr32(falcon, 0xac4 + (port * 8),
+ extra & (BIT(rem * 8) - 1));
+ }
+}
+
+static const u32 EMEM_START_ADDR = 0x1000000;
+
static void
nvkm_falcon_v1_load_dmem(struct nvkm_falcon *falcon, void *data, u32 start,
u32 size, u8 port)
u8 rem = size % 4;
int i;
+ if (start >= EMEM_START_ADDR && falcon->has_emem)
+ return nvkm_falcon_v1_load_emem(falcon, data,
+ start - EMEM_START_ADDR, size,
+ port);
+
size -= rem;
- nvkm_falcon_wr32(falcon, 0x1c0 + (port * 16), start | (0x1 << 24));
+ nvkm_falcon_wr32(falcon, 0x1c0 + (port * 8), start | (0x1 << 24));
for (i = 0; i < size / 4; i++)
- nvkm_falcon_wr32(falcon, 0x1c4, ((u32 *)data)[i]);
+ nvkm_falcon_wr32(falcon, 0x1c4 + (port * 8), ((u32 *)data)[i]);
/*
- * If size is not a multiple of 4, mask the last work to ensure garbage
- * does not get read
+ * If size is not a multiple of 4, mask the last word to ensure garbage
+ * does not get written
*/
if (rem) {
u32 extra = ((u32 *)data)[i];
- nvkm_falcon_wr32(falcon, 0x1c4, extra & (BIT(rem * 8) - 1));
+ nvkm_falcon_wr32(falcon, 0x1c4 + (port * 8),
+ extra & (BIT(rem * 8) - 1));
+ }
+}
+
+static void
+nvkm_falcon_v1_read_emem(struct nvkm_falcon *falcon, u32 start, u32 size,
+ u8 port, void *data)
+{
+ u8 rem = size % 4;
+ int i;
+
+ size -= rem;
+
+ nvkm_falcon_wr32(falcon, 0xac0 + (port * 8), start | (0x1 << 25));
+ for (i = 0; i < size / 4; i++)
+ ((u32 *)data)[i] = nvkm_falcon_rd32(falcon, 0xac4 + (port * 8));
+
+ /*
+ * If size is not a multiple of 4, mask the last word to ensure garbage
+ * does not get read
+ */
+ if (rem) {
+ u32 extra = nvkm_falcon_rd32(falcon, 0xac4 + (port * 8));
+
+ for (i = size; i < size + rem; i++) {
+ ((u8 *)data)[i] = (u8)(extra & 0xff);
+ extra >>= 8;
+ }
}
}
u8 rem = size % 4;
int i;
+ if (start >= EMEM_START_ADDR && falcon->has_emem)
+ return nvkm_falcon_v1_read_emem(falcon, start - EMEM_START_ADDR,
+ size, port, data);
+
size -= rem;
- nvkm_falcon_wr32(falcon, 0x1c0 + (port * 16), start | (0x1 << 25));
+ nvkm_falcon_wr32(falcon, 0x1c0 + (port * 8), start | (0x1 << 25));
for (i = 0; i < size / 4; i++)
- ((u32 *)data)[i] = nvkm_falcon_rd32(falcon, 0x1c4);
+ ((u32 *)data)[i] = nvkm_falcon_rd32(falcon, 0x1c4 + (port * 8));
/*
- * If size is not a multiple of 4, mask the last work to ensure garbage
+ * If size is not a multiple of 4, mask the last word to ensure garbage
* does not get read
*/
if (rem) {
- u32 extra = nvkm_falcon_rd32(falcon, 0x1c4);
+ u32 extra = nvkm_falcon_rd32(falcon, 0x1c4 + (port * 8));
for (i = size; i < size + rem; i++) {
((u8 *)data)[i] = (u8)(extra & 0xff);
nvkm_falcon_v1_bind_context(struct nvkm_falcon *falcon, struct nvkm_gpuobj *ctx)
{
u32 inst_loc;
+ u32 fbif;
/* disable instance block binding */
if (ctx == NULL) {
return;
}
+ switch (falcon->owner->index) {
+ case NVKM_ENGINE_NVENC0:
+ case NVKM_ENGINE_NVENC1:
+ case NVKM_ENGINE_NVENC2:
+ fbif = 0x800;
+ break;
+ case NVKM_SUBDEV_PMU:
+ fbif = 0xe00;
+ break;
+ default:
+ fbif = 0x600;
+ break;
+ }
+
nvkm_falcon_wr32(falcon, 0x10c, 0x1);
/* setup apertures - virtual */
- nvkm_falcon_wr32(falcon, 0xe00 + 4 * FALCON_DMAIDX_UCODE, 0x4);
- nvkm_falcon_wr32(falcon, 0xe00 + 4 * FALCON_DMAIDX_VIRT, 0x0);
+ nvkm_falcon_wr32(falcon, fbif + 4 * FALCON_DMAIDX_UCODE, 0x4);
+ nvkm_falcon_wr32(falcon, fbif + 4 * FALCON_DMAIDX_VIRT, 0x0);
/* setup apertures - physical */
- nvkm_falcon_wr32(falcon, 0xe00 + 4 * FALCON_DMAIDX_PHYS_VID, 0x4);
- nvkm_falcon_wr32(falcon, 0xe00 + 4 * FALCON_DMAIDX_PHYS_SYS_COH, 0x5);
- nvkm_falcon_wr32(falcon, 0xe00 + 4 * FALCON_DMAIDX_PHYS_SYS_NCOH, 0x6);
+ nvkm_falcon_wr32(falcon, fbif + 4 * FALCON_DMAIDX_PHYS_VID, 0x4);
+ nvkm_falcon_wr32(falcon, fbif + 4 * FALCON_DMAIDX_PHYS_SYS_COH, 0x5);
+ nvkm_falcon_wr32(falcon, fbif + 4 * FALCON_DMAIDX_PHYS_SYS_NCOH, 0x6);
/* Set context */
switch (nvkm_memory_target(ctx->memory)) {
case NVKM_MEM_TARGET_VRAM: inst_loc = 0; break;
+ case NVKM_MEM_TARGET_HOST: inst_loc = 2; break;
case NVKM_MEM_TARGET_NCOH: inst_loc = 3; break;
default:
WARN_ON(1);
/* Enable context */
nvkm_falcon_mask(falcon, 0x048, 0x1, 0x1);
- nvkm_falcon_wr32(falcon, 0x480,
+ nvkm_falcon_wr32(falcon, 0x054,
((ctx->addr >> 12) & 0xfffffff) |
(inst_loc << 28) | (1 << 30));
+
+ nvkm_falcon_mask(falcon, 0x090, 0x10000, 0x10000);
+ nvkm_falcon_mask(falcon, 0x0a4, 0x8, 0x8);
}
static void
nvkm-y += nvkm/subdev/fb/mcp77.o
nvkm-y += nvkm/subdev/fb/mcp89.o
nvkm-y += nvkm/subdev/fb/gf100.o
+nvkm-y += nvkm/subdev/fb/gf108.o
nvkm-y += nvkm/subdev/fb/gk104.o
nvkm-y += nvkm/subdev/fb/gk20a.o
nvkm-y += nvkm/subdev/fb/gm107.o
nvkm-y += nvkm/subdev/fb/ramgt215.o
nvkm-y += nvkm/subdev/fb/rammcp77.o
nvkm-y += nvkm/subdev/fb/ramgf100.o
+nvkm-y += nvkm/subdev/fb/ramgf108.o
nvkm-y += nvkm/subdev/fb/ramgk104.o
nvkm-y += nvkm/subdev/fb/ramgm107.o
+nvkm-y += nvkm/subdev/fb/ramgm200.o
nvkm-y += nvkm/subdev/fb/ramgp100.o
nvkm-y += nvkm/subdev/fb/sddr2.o
nvkm-y += nvkm/subdev/fb/sddr3.o
--- /dev/null
+/*
+ * Copyright 2017 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs <bskeggs@redhat.com>
+ */
+#include "gf100.h"
+#include "ram.h"
+
+static const struct nvkm_fb_func
+gf108_fb = {
+ .dtor = gf100_fb_dtor,
+ .oneinit = gf100_fb_oneinit,
+ .init = gf100_fb_init,
+ .init_page = gf100_fb_init_page,
+ .intr = gf100_fb_intr,
+ .ram_new = gf108_ram_new,
+ .memtype_valid = gf100_fb_memtype_valid,
+};
+
+int
+gf108_fb_new(struct nvkm_device *device, int index, struct nvkm_fb **pfb)
+{
+ return gf100_fb_new_(&gf108_fb, device, index, pfb);
+}
.init = gm200_fb_init,
.init_page = gm200_fb_init_page,
.intr = gf100_fb_intr,
- .ram_new = gm107_ram_new,
+ .ram_new = gm200_ram_new,
.memtype_valid = gf100_fb_memtype_valid,
};
void nv50_ram_put(struct nvkm_ram *, struct nvkm_mem **);
void __nv50_ram_put(struct nvkm_ram *, struct nvkm_mem *);
+int gf100_ram_new_(const struct nvkm_ram_func *, struct nvkm_fb *,
+ struct nvkm_ram **);
int gf100_ram_ctor(const struct nvkm_ram_func *, struct nvkm_fb *,
- u32, struct nvkm_ram *);
+ struct nvkm_ram *);
+u32 gf100_ram_probe_fbp(const struct nvkm_ram_func *,
+ struct nvkm_device *, int, int *);
+u32 gf100_ram_probe_fbp_amount(const struct nvkm_ram_func *, u32,
+ struct nvkm_device *, int, int *);
+u32 gf100_ram_probe_fbpa_amount(struct nvkm_device *, int);
int gf100_ram_get(struct nvkm_ram *, u64, u32, u32, u32, struct nvkm_mem **);
void gf100_ram_put(struct nvkm_ram *, struct nvkm_mem **);
+int gf100_ram_init(struct nvkm_ram *);
+int gf100_ram_calc(struct nvkm_ram *, u32);
+int gf100_ram_prog(struct nvkm_ram *);
+void gf100_ram_tidy(struct nvkm_ram *);
+
+u32 gf108_ram_probe_fbp_amount(const struct nvkm_ram_func *, u32,
+ struct nvkm_device *, int, int *);
+
+int gk104_ram_new_(const struct nvkm_ram_func *, struct nvkm_fb *,
+ struct nvkm_ram **);
+void *gk104_ram_dtor(struct nvkm_ram *);
+int gk104_ram_init(struct nvkm_ram *);
+int gk104_ram_calc(struct nvkm_ram *, u32);
+int gk104_ram_prog(struct nvkm_ram *);
+void gk104_ram_tidy(struct nvkm_ram *);
+
+u32 gm107_ram_probe_fbp(const struct nvkm_ram_func *,
+ struct nvkm_device *, int, int *);
-int gk104_ram_ctor(struct nvkm_fb *, struct nvkm_ram **, u32);
-int gk104_ram_init(struct nvkm_ram *ram);
+u32 gm200_ram_probe_fbp_amount(const struct nvkm_ram_func *, u32,
+ struct nvkm_device *, int, int *);
/* RAM type-specific MR calculation routines */
int nvkm_sddr2_calc(struct nvkm_ram *);
int gt215_ram_new(struct nvkm_fb *, struct nvkm_ram **);
int mcp77_ram_new(struct nvkm_fb *, struct nvkm_ram **);
int gf100_ram_new(struct nvkm_fb *, struct nvkm_ram **);
+int gf108_ram_new(struct nvkm_fb *, struct nvkm_ram **);
int gk104_ram_new(struct nvkm_fb *, struct nvkm_ram **);
int gm107_ram_new(struct nvkm_fb *, struct nvkm_ram **);
+int gm200_ram_new(struct nvkm_fb *, struct nvkm_ram **);
int gp100_ram_new(struct nvkm_fb *, struct nvkm_ram **);
#endif
}
}
-static int
+int
gf100_ram_calc(struct nvkm_ram *base, u32 freq)
{
struct gf100_ram *ram = gf100_ram(base);
return 0;
}
-static int
+int
gf100_ram_prog(struct nvkm_ram *base)
{
struct gf100_ram *ram = gf100_ram(base);
return 0;
}
-static void
+void
gf100_ram_tidy(struct nvkm_ram *base)
{
struct gf100_ram *ram = gf100_ram(base);
return 0;
}
-static int
+int
gf100_ram_init(struct nvkm_ram *base)
{
static const u8 train0[] = {
return 0;
}
-static const struct nvkm_ram_func
-gf100_ram_func = {
- .init = gf100_ram_init,
- .get = gf100_ram_get,
- .put = gf100_ram_put,
- .calc = gf100_ram_calc,
- .prog = gf100_ram_prog,
- .tidy = gf100_ram_tidy,
-};
+u32
+gf100_ram_probe_fbpa_amount(struct nvkm_device *device, int fbpa)
+{
+ return nvkm_rd32(device, 0x11020c + (fbpa * 0x1000));
+}
+
+u32
+gf100_ram_probe_fbp_amount(const struct nvkm_ram_func *func, u32 fbpao,
+ struct nvkm_device *device, int fbp, int *pltcs)
+{
+ if (!(fbpao & BIT(fbp))) {
+ *pltcs = 1;
+ return func->probe_fbpa_amount(device, fbp);
+ }
+ return 0;
+}
+
+u32
+gf100_ram_probe_fbp(const struct nvkm_ram_func *func,
+ struct nvkm_device *device, int fbp, int *pltcs)
+{
+ u32 fbpao = nvkm_rd32(device, 0x022554);
+ return func->probe_fbp_amount(func, fbpao, device, fbp, pltcs);
+}
int
gf100_ram_ctor(const struct nvkm_ram_func *func, struct nvkm_fb *fb,
- u32 maskaddr, struct nvkm_ram *ram)
+ struct nvkm_ram *ram)
{
struct nvkm_subdev *subdev = &fb->subdev;
struct nvkm_device *device = subdev->device;
struct nvkm_bios *bios = device->bios;
const u32 rsvd_head = ( 256 * 1024); /* vga memory */
const u32 rsvd_tail = (1024 * 1024); /* vbios etc */
- u32 parts = nvkm_rd32(device, 0x022438);
- u32 pmask = nvkm_rd32(device, maskaddr);
- u64 bsize = (u64)nvkm_rd32(device, 0x10f20c) << 20;
- u64 psize, size = 0;
enum nvkm_ram_type type = nvkm_fb_bios_memtype(bios);
- bool uniform = true;
- int ret, i;
-
- nvkm_debug(subdev, "100800: %08x\n", nvkm_rd32(device, 0x100800));
- nvkm_debug(subdev, "parts %08x mask %08x\n", parts, pmask);
-
- /* read amount of vram attached to each memory controller */
- for (i = 0; i < parts; i++) {
- if (pmask & (1 << i))
- continue;
-
- psize = (u64)nvkm_rd32(device, 0x11020c + (i * 0x1000)) << 20;
- if (psize != bsize) {
- if (psize < bsize)
- bsize = psize;
- uniform = false;
+ u32 fbps = nvkm_rd32(device, 0x022438);
+ u64 total = 0, lcomm = ~0, lower, ubase, usize;
+ int ret, fbp, ltcs, ltcn = 0;
+
+ nvkm_debug(subdev, "%d FBP(s)\n", fbps);
+ for (fbp = 0; fbp < fbps; fbp++) {
+ u32 size = func->probe_fbp(func, device, fbp, <cs);
+ if (size) {
+ nvkm_debug(subdev, "FBP %d: %4d MiB, %d LTC(s)\n",
+ fbp, size, ltcs);
+ lcomm = min(lcomm, (u64)(size / ltcs) << 20);
+ total += size << 20;
+ ltcn += ltcs;
+ } else {
+ nvkm_debug(subdev, "FBP %d: disabled\n", fbp);
}
-
- nvkm_debug(subdev, "%d: %d MiB\n", i, (u32)(psize >> 20));
- size += psize;
}
- ret = nvkm_ram_ctor(func, fb, type, size, 0, ram);
+ lower = lcomm * ltcn;
+ ubase = lcomm + func->upper;
+ usize = total - lower;
+
+ nvkm_debug(subdev, "Lower: %4lld MiB @ %010llx\n", lower >> 20, 0ULL);
+ nvkm_debug(subdev, "Upper: %4lld MiB @ %010llx\n", usize >> 20, ubase);
+ nvkm_debug(subdev, "Total: %4lld MiB\n", total >> 20);
+
+ ret = nvkm_ram_ctor(func, fb, type, total, 0, ram);
if (ret)
return ret;
nvkm_mm_fini(&ram->vram);
- /* if all controllers have the same amount attached, there's no holes */
- if (uniform) {
+ /* Some GPUs are in what's known as a "mixed memory" configuration.
+ *
+ * This is either where some FBPs have more memory than the others,
+ * or where LTCs have been disabled on a FBP.
+ */
+ if (lower != total) {
+ /* The common memory amount is addressed normally. */
ret = nvkm_mm_init(&ram->vram, rsvd_head >> NVKM_RAM_MM_SHIFT,
- (size - rsvd_head - rsvd_tail) >>
- NVKM_RAM_MM_SHIFT, 1);
+ (lower - rsvd_head) >> NVKM_RAM_MM_SHIFT, 1);
if (ret)
return ret;
- } else {
- /* otherwise, address lowest common amount from 0GiB */
- ret = nvkm_mm_init(&ram->vram, rsvd_head >> NVKM_RAM_MM_SHIFT,
- ((bsize * parts) - rsvd_head) >>
- NVKM_RAM_MM_SHIFT, 1);
+
+ /* And the rest is much higher in the physical address
+ * space, and may not be usable for certain operations.
+ */
+ ret = nvkm_mm_init(&ram->vram, ubase >> NVKM_RAM_MM_SHIFT,
+ (usize - rsvd_tail) >> NVKM_RAM_MM_SHIFT, 1);
if (ret)
return ret;
-
- /* and the rest starting from (8GiB + common_size) */
- ret = nvkm_mm_init(&ram->vram, (0x0200000000ULL + bsize) >>
- NVKM_RAM_MM_SHIFT,
- (size - (bsize * parts) - rsvd_tail) >>
+ } else {
+ /* GPUs without mixed-memory are a lot nicer... */
+ ret = nvkm_mm_init(&ram->vram, rsvd_head >> NVKM_RAM_MM_SHIFT,
+ (total - rsvd_head - rsvd_tail) >>
NVKM_RAM_MM_SHIFT, 1);
if (ret)
return ret;
}
int
-gf100_ram_new(struct nvkm_fb *fb, struct nvkm_ram **pram)
+gf100_ram_new_(const struct nvkm_ram_func *func,
+ struct nvkm_fb *fb, struct nvkm_ram **pram)
{
struct nvkm_subdev *subdev = &fb->subdev;
struct nvkm_bios *bios = subdev->device->bios;
return -ENOMEM;
*pram = &ram->base;
- ret = gf100_ram_ctor(&gf100_ram_func, fb, 0x022554, &ram->base);
+ ret = gf100_ram_ctor(func, fb, &ram->base);
if (ret)
return ret;
ram->fuc.r_0x13d8f4 = ramfuc_reg(0x13d8f4);
return 0;
}
+
+static const struct nvkm_ram_func
+gf100_ram = {
+ .upper = 0x0200000000,
+ .probe_fbp = gf100_ram_probe_fbp,
+ .probe_fbp_amount = gf100_ram_probe_fbp_amount,
+ .probe_fbpa_amount = gf100_ram_probe_fbpa_amount,
+ .init = gf100_ram_init,
+ .get = gf100_ram_get,
+ .put = gf100_ram_put,
+ .calc = gf100_ram_calc,
+ .prog = gf100_ram_prog,
+ .tidy = gf100_ram_tidy,
+};
+
+int
+gf100_ram_new(struct nvkm_fb *fb, struct nvkm_ram **pram)
+{
+ return gf100_ram_new_(&gf100_ram, fb, pram);
+}
--- /dev/null
+/*
+ * Copyright 2017 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs <bskeggs@redhat.com>
+ */
+#include "ram.h"
+
+u32
+gf108_ram_probe_fbp_amount(const struct nvkm_ram_func *func, u32 fbpao,
+ struct nvkm_device *device, int fbp, int *pltcs)
+{
+ u32 fbpt = nvkm_rd32(device, 0x022438);
+ u32 fbpat = nvkm_rd32(device, 0x02243c);
+ u32 fbpas = fbpat / fbpt;
+ u32 fbpa = fbp * fbpas;
+ u32 size = 0;
+ while (fbpas--) {
+ if (!(fbpao & BIT(fbpa)))
+ size += func->probe_fbpa_amount(device, fbpa);
+ fbpa++;
+ }
+ *pltcs = 1;
+ return size;
+}
+
+static const struct nvkm_ram_func
+gf108_ram = {
+ .upper = 0x0200000000,
+ .probe_fbp = gf100_ram_probe_fbp,
+ .probe_fbp_amount = gf108_ram_probe_fbp_amount,
+ .probe_fbpa_amount = gf100_ram_probe_fbpa_amount,
+ .init = gf100_ram_init,
+ .get = gf100_ram_get,
+ .put = gf100_ram_put,
+ .calc = gf100_ram_calc,
+ .prog = gf100_ram_prog,
+ .tidy = gf100_ram_tidy,
+};
+
+int
+gf108_ram_new(struct nvkm_fb *fb, struct nvkm_ram **pram)
+{
+ return gf100_ram_new_(&gf108_ram, fb, pram);
+}
return ret;
}
-static int
+int
gk104_ram_calc(struct nvkm_ram *base, u32 freq)
{
struct gk104_ram *ram = gk104_ram(base);
nvkm_mask(device, 0x10f444, mask, data);
}
-static int
+int
gk104_ram_prog(struct nvkm_ram *base)
{
struct gk104_ram *ram = gk104_ram(base);
return (ram->base.next == &ram->base.xition);
}
-static void
+void
gk104_ram_tidy(struct nvkm_ram *base)
{
struct gk104_ram *ram = gk104_ram(base);
return ret;
}
-static void *
+void *
gk104_ram_dtor(struct nvkm_ram *base)
{
struct gk104_ram *ram = gk104_ram(base);
return ram;
}
-static const struct nvkm_ram_func
-gk104_ram_func = {
- .dtor = gk104_ram_dtor,
- .init = gk104_ram_init,
- .get = gf100_ram_get,
- .put = gf100_ram_put,
- .calc = gk104_ram_calc,
- .prog = gk104_ram_prog,
- .tidy = gk104_ram_tidy,
-};
-
int
-gk104_ram_new(struct nvkm_fb *fb, struct nvkm_ram **pram)
-{
- return gk104_ram_ctor(fb, pram, 0x022554);
-}
-
-int
-gk104_ram_ctor(struct nvkm_fb *fb, struct nvkm_ram **pram, u32 maskaddr)
+gk104_ram_new_(const struct nvkm_ram_func *func, struct nvkm_fb *fb,
+ struct nvkm_ram **pram)
{
struct nvkm_subdev *subdev = &fb->subdev;
struct nvkm_device *device = subdev->device;
struct nvkm_bios *bios = device->bios;
- struct nvkm_gpio *gpio = device->gpio;
- struct dcb_gpio_func func;
+ struct dcb_gpio_func gpio;
struct gk104_ram *ram;
int ret, i;
u8 ramcfg = nvbios_ramcfg_index(subdev);
return -ENOMEM;
*pram = &ram->base;
- ret = gf100_ram_ctor(&gk104_ram_func, fb, maskaddr, &ram->base);
+ ret = gf100_ram_ctor(func, fb, &ram->base);
if (ret)
return ret;
}
/* lookup memory voltage gpios */
- ret = nvkm_gpio_find(gpio, 0, 0x18, DCB_GPIO_UNUSED, &func);
+ ret = nvkm_gpio_find(device->gpio, 0, 0x18, DCB_GPIO_UNUSED, &gpio);
if (ret == 0) {
- ram->fuc.r_gpioMV = ramfuc_reg(0x00d610 + (func.line * 0x04));
- ram->fuc.r_funcMV[0] = (func.log[0] ^ 2) << 12;
- ram->fuc.r_funcMV[1] = (func.log[1] ^ 2) << 12;
+ ram->fuc.r_gpioMV = ramfuc_reg(0x00d610 + (gpio.line * 0x04));
+ ram->fuc.r_funcMV[0] = (gpio.log[0] ^ 2) << 12;
+ ram->fuc.r_funcMV[1] = (gpio.log[1] ^ 2) << 12;
}
- ret = nvkm_gpio_find(gpio, 0, 0x2e, DCB_GPIO_UNUSED, &func);
+ ret = nvkm_gpio_find(device->gpio, 0, 0x2e, DCB_GPIO_UNUSED, &gpio);
if (ret == 0) {
- ram->fuc.r_gpio2E = ramfuc_reg(0x00d610 + (func.line * 0x04));
- ram->fuc.r_func2E[0] = (func.log[0] ^ 2) << 12;
- ram->fuc.r_func2E[1] = (func.log[1] ^ 2) << 12;
+ ram->fuc.r_gpio2E = ramfuc_reg(0x00d610 + (gpio.line * 0x04));
+ ram->fuc.r_func2E[0] = (gpio.log[0] ^ 2) << 12;
+ ram->fuc.r_func2E[1] = (gpio.log[1] ^ 2) << 12;
}
ram->fuc.r_gpiotrig = ramfuc_reg(0x00d604);
ram->fuc.r_0x100750 = ramfuc_reg(0x100750);
return 0;
}
+
+static const struct nvkm_ram_func
+gk104_ram = {
+ .upper = 0x0200000000,
+ .probe_fbp = gf100_ram_probe_fbp,
+ .probe_fbp_amount = gf108_ram_probe_fbp_amount,
+ .probe_fbpa_amount = gf100_ram_probe_fbpa_amount,
+ .dtor = gk104_ram_dtor,
+ .init = gk104_ram_init,
+ .get = gf100_ram_get,
+ .put = gf100_ram_put,
+ .calc = gk104_ram_calc,
+ .prog = gk104_ram_prog,
+ .tidy = gk104_ram_tidy,
+};
+
+int
+gk104_ram_new(struct nvkm_fb *fb, struct nvkm_ram **pram)
+{
+ return gk104_ram_new_(&gk104_ram, fb, pram);
+}
*/
#include "ram.h"
+u32
+gm107_ram_probe_fbp(const struct nvkm_ram_func *func,
+ struct nvkm_device *device, int fbp, int *pltcs)
+{
+ u32 fbpao = nvkm_rd32(device, 0x021c14);
+ return func->probe_fbp_amount(func, fbpao, device, fbp, pltcs);
+}
+
+static const struct nvkm_ram_func
+gm107_ram = {
+ .upper = 0x1000000000,
+ .probe_fbp = gm107_ram_probe_fbp,
+ .probe_fbp_amount = gf108_ram_probe_fbp_amount,
+ .probe_fbpa_amount = gf100_ram_probe_fbpa_amount,
+ .dtor = gk104_ram_dtor,
+ .init = gk104_ram_init,
+ .get = gf100_ram_get,
+ .put = gf100_ram_put,
+ .calc = gk104_ram_calc,
+ .prog = gk104_ram_prog,
+ .tidy = gk104_ram_tidy,
+};
+
int
gm107_ram_new(struct nvkm_fb *fb, struct nvkm_ram **pram)
{
- return gk104_ram_ctor(fb, pram, 0x021c14);
+ return gk104_ram_new_(&gm107_ram, fb, pram);
}
--- /dev/null
+/*
+ * Copyright 2017 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs <bskeggs@redhat.com>
+ */
+#include "ram.h"
+
+u32
+gm200_ram_probe_fbp_amount(const struct nvkm_ram_func *func, u32 fbpao,
+ struct nvkm_device *device, int fbp, int *pltcs)
+{
+ u32 ltcs = nvkm_rd32(device, 0x022450);
+ u32 fbpas = nvkm_rd32(device, 0x022458);
+ u32 fbpa = fbp * fbpas;
+ u32 size = 0;
+ if (!(nvkm_rd32(device, 0x021d38) & BIT(fbp))) {
+ u32 ltco = nvkm_rd32(device, 0x021d70 + (fbp * 4));
+ u32 ltcm = ~ltco & ((1 << ltcs) - 1);
+
+ while (fbpas--) {
+ if (!(fbpao & (1 << fbpa)))
+ size += func->probe_fbpa_amount(device, fbpa);
+ fbpa++;
+ }
+
+ *pltcs = hweight32(ltcm);
+ }
+ return size;
+}
+
+static const struct nvkm_ram_func
+gm200_ram = {
+ .upper = 0x1000000000,
+ .probe_fbp = gm107_ram_probe_fbp,
+ .probe_fbp_amount = gm200_ram_probe_fbp_amount,
+ .probe_fbpa_amount = gf100_ram_probe_fbpa_amount,
+ .dtor = gk104_ram_dtor,
+ .init = gk104_ram_init,
+ .get = gf100_ram_get,
+ .put = gf100_ram_put,
+ .calc = gk104_ram_calc,
+ .prog = gk104_ram_prog,
+ .tidy = gk104_ram_tidy,
+};
+
+int
+gm200_ram_new(struct nvkm_fb *fb, struct nvkm_ram **pram)
+{
+ return gk104_ram_new_(&gm200_ram, fb, pram);
+}
return 0;
}
+static u32
+gp100_ram_probe_fbpa(struct nvkm_device *device, int fbpa)
+{
+ return nvkm_rd32(device, 0x90020c + (fbpa * 0x4000));
+}
+
static const struct nvkm_ram_func
-gp100_ram_func = {
+gp100_ram = {
+ .upper = 0x1000000000,
+ .probe_fbp = gm107_ram_probe_fbp,
+ .probe_fbp_amount = gm200_ram_probe_fbp_amount,
+ .probe_fbpa_amount = gp100_ram_probe_fbpa,
.init = gp100_ram_init,
.get = gf100_ram_get,
.put = gf100_ram_put,
gp100_ram_new(struct nvkm_fb *fb, struct nvkm_ram **pram)
{
struct nvkm_ram *ram;
- struct nvkm_subdev *subdev = &fb->subdev;
- struct nvkm_device *device = subdev->device;
- enum nvkm_ram_type type = nvkm_fb_bios_memtype(device->bios);
- const u32 rsvd_head = ( 256 * 1024); /* vga memory */
- const u32 rsvd_tail = (1024 * 1024); /* vbios etc */
- u32 fbpa_num = nvkm_rd32(device, 0x02243c), fbpa;
- u32 fbio_opt = nvkm_rd32(device, 0x021c14);
- u64 part, size = 0, comm = ~0ULL;
- bool mixed = false;
- int ret;
-
- nvkm_debug(subdev, "02243c: %08x\n", fbpa_num);
- nvkm_debug(subdev, "021c14: %08x\n", fbio_opt);
- for (fbpa = 0; fbpa < fbpa_num; fbpa++) {
- if (!(fbio_opt & (1 << fbpa))) {
- part = nvkm_rd32(device, 0x90020c + (fbpa * 0x4000));
- nvkm_debug(subdev, "fbpa %02x: %lld MiB\n", fbpa, part);
- part = part << 20;
- if (part != comm) {
- if (comm != ~0ULL)
- mixed = true;
- comm = min(comm, part);
- }
- size = size + part;
- }
- }
-
- ret = nvkm_ram_new_(&gp100_ram_func, fb, type, size, 0, &ram);
- *pram = ram;
- if (ret)
- return ret;
- nvkm_mm_fini(&ram->vram);
+ if (!(ram = *pram = kzalloc(sizeof(*ram), GFP_KERNEL)))
+ return -ENOMEM;
- if (mixed) {
- ret = nvkm_mm_init(&ram->vram, rsvd_head >> NVKM_RAM_MM_SHIFT,
- ((comm * fbpa_num) - rsvd_head) >>
- NVKM_RAM_MM_SHIFT, 1);
- if (ret)
- return ret;
+ return gf100_ram_ctor(&gp100_ram, fb, ram);
- ret = nvkm_mm_init(&ram->vram, (0x1000000000ULL + comm) >>
- NVKM_RAM_MM_SHIFT,
- (size - (comm * fbpa_num) - rsvd_tail) >>
- NVKM_RAM_MM_SHIFT, 1);
- if (ret)
- return ret;
- } else {
- ret = nvkm_mm_init(&ram->vram, rsvd_head >> NVKM_RAM_MM_SHIFT,
- (size - rsvd_head - rsvd_tail) >>
- NVKM_RAM_MM_SHIFT, 1);
- if (ret)
- return ret;
- }
-
- return 0;
}
static int
anx9805_aux_xfer(struct nvkm_i2c_aux *base, bool retry,
- u8 type, u32 addr, u8 *data, u8 size)
+ u8 type, u32 addr, u8 *data, u8 *size)
{
struct anx9805_aux *aux = anx9805_aux(base);
struct anx9805_pad *pad = aux->pad;
u8 buf[16] = {};
u8 tmp;
- AUX_DBG(&aux->base, "%02x %05x %d", type, addr, size);
+ AUX_DBG(&aux->base, "%02x %05x %d", type, addr, *size);
tmp = nvkm_rdi2cr(adap, pad->addr, 0x07) & ~0x04;
nvkm_wri2cr(adap, pad->addr, 0x07, tmp | 0x04);
nvkm_wri2cr(adap, aux->addr, 0xe4, 0x80);
if (!(type & 1)) {
- memcpy(buf, data, size);
+ memcpy(buf, data, *size);
AUX_DBG(&aux->base, "%16ph", buf);
- for (i = 0; i < size; i++)
+ for (i = 0; i < *size; i++)
nvkm_wri2cr(adap, aux->addr, 0xf0 + i, buf[i]);
}
- nvkm_wri2cr(adap, aux->addr, 0xe5, ((size - 1) << 4) | type);
+ nvkm_wri2cr(adap, aux->addr, 0xe5, ((*size - 1) << 4) | type);
nvkm_wri2cr(adap, aux->addr, 0xe6, (addr & 0x000ff) >> 0);
nvkm_wri2cr(adap, aux->addr, 0xe7, (addr & 0x0ff00) >> 8);
nvkm_wri2cr(adap, aux->addr, 0xe8, (addr & 0xf0000) >> 16);
}
if (type & 1) {
- for (i = 0; i < size; i++)
+ for (i = 0; i < *size; i++)
buf[i] = nvkm_rdi2cr(adap, aux->addr, 0xf0 + i);
AUX_DBG(&aux->base, "%16ph", buf);
- memcpy(data, buf, size);
+ memcpy(data, buf, *size);
}
ret = 0;
if (mcnt || remaining > 16)
cmd |= 4; /* MOT */
- ret = aux->func->xfer(aux, true, cmd, msg->addr, ptr, cnt);
+ ret = aux->func->xfer(aux, true, cmd, msg->addr, ptr, &cnt);
if (ret < 0) {
nvkm_i2c_aux_release(aux);
return ret;
int
nvkm_i2c_aux_xfer(struct nvkm_i2c_aux *aux, bool retry, u8 type,
- u32 addr, u8 *data, u8 size)
+ u32 addr, u8 *data, u8 *size)
{
return aux->func->xfer(aux, retry, type, addr, data, size);
}
struct nvkm_i2c_aux_func {
int (*xfer)(struct nvkm_i2c_aux *, bool retry, u8 type,
- u32 addr, u8 *data, u8 size);
+ u32 addr, u8 *data, u8 *size);
int (*lnk_ctl)(struct nvkm_i2c_aux *, int link_nr, int link_bw,
bool enhanced_framing);
};
int id, struct nvkm_i2c_aux **);
void nvkm_i2c_aux_del(struct nvkm_i2c_aux **);
int nvkm_i2c_aux_xfer(struct nvkm_i2c_aux *, bool retry, u8 type,
- u32 addr, u8 *data, u8 size);
+ u32 addr, u8 *data, u8 *size);
int g94_i2c_aux_new(struct nvkm_i2c_pad *, int, u8, struct nvkm_i2c_aux **);
int gm200_i2c_aux_new(struct nvkm_i2c_pad *, int, u8, struct nvkm_i2c_aux **);
static int
g94_i2c_aux_xfer(struct nvkm_i2c_aux *obj, bool retry,
- u8 type, u32 addr, u8 *data, u8 size)
+ u8 type, u32 addr, u8 *data, u8 *size)
{
struct g94_i2c_aux *aux = g94_i2c_aux(obj);
struct nvkm_device *device = aux->base.pad->i2c->subdev.device;
u32 xbuf[4] = {};
int ret, i;
- AUX_TRACE(&aux->base, "%d: %08x %d", type, addr, size);
+ AUX_TRACE(&aux->base, "%d: %08x %d", type, addr, *size);
ret = g94_i2c_aux_init(aux);
if (ret < 0)
}
if (!(type & 1)) {
- memcpy(xbuf, data, size);
+ memcpy(xbuf, data, *size);
for (i = 0; i < 16; i += 4) {
AUX_TRACE(&aux->base, "wr %08x", xbuf[i / 4]);
nvkm_wr32(device, 0x00e4c0 + base + i, xbuf[i / 4]);
ctrl = nvkm_rd32(device, 0x00e4e4 + base);
ctrl &= ~0x0001f0ff;
ctrl |= type << 12;
- ctrl |= size - 1;
+ ctrl |= *size - 1;
nvkm_wr32(device, 0x00e4e0 + base, addr);
/* (maybe) retry transaction a number of times on failure... */
xbuf[i / 4] = nvkm_rd32(device, 0x00e4d0 + base + i);
AUX_TRACE(&aux->base, "rd %08x", xbuf[i / 4]);
}
- memcpy(data, xbuf, size);
+ memcpy(data, xbuf, *size);
+ *size = stat & 0x0000001f;
}
out:
static int
gm200_i2c_aux_xfer(struct nvkm_i2c_aux *obj, bool retry,
- u8 type, u32 addr, u8 *data, u8 size)
+ u8 type, u32 addr, u8 *data, u8 *size)
{
struct gm200_i2c_aux *aux = gm200_i2c_aux(obj);
struct nvkm_device *device = aux->base.pad->i2c->subdev.device;
u32 xbuf[4] = {};
int ret, i;
- AUX_TRACE(&aux->base, "%d: %08x %d", type, addr, size);
+ AUX_TRACE(&aux->base, "%d: %08x %d", type, addr, *size);
ret = gm200_i2c_aux_init(aux);
if (ret < 0)
}
if (!(type & 1)) {
- memcpy(xbuf, data, size);
+ memcpy(xbuf, data, *size);
for (i = 0; i < 16; i += 4) {
AUX_TRACE(&aux->base, "wr %08x", xbuf[i / 4]);
nvkm_wr32(device, 0x00d930 + base + i, xbuf[i / 4]);
ctrl = nvkm_rd32(device, 0x00d954 + base);
ctrl &= ~0x0001f0ff;
ctrl |= type << 12;
- ctrl |= size - 1;
+ ctrl |= *size - 1;
nvkm_wr32(device, 0x00d950 + base, addr);
/* (maybe) retry transaction a number of times on failure... */
xbuf[i / 4] = nvkm_rd32(device, 0x00d940 + base + i);
AUX_TRACE(&aux->base, "rd %08x", xbuf[i / 4]);
}
- memcpy(data, xbuf, size);
+ memcpy(data, xbuf, *size);
+ *size = stat & 0x0000001f;
}
out:
u32 addr = nvkm_rd32(device, 0x122120 + (i * 0x0400));
u32 data = nvkm_rd32(device, 0x122124 + (i * 0x0400));
u32 stat = nvkm_rd32(device, 0x122128 + (i * 0x0400));
- nvkm_error(ibus, "HUB%d: %06x %08x (%08x)\n", i, addr, data, stat);
+ nvkm_debug(ibus, "HUB%d: %06x %08x (%08x)\n", i, addr, data, stat);
nvkm_mask(device, 0x122128 + (i * 0x0400), 0x00000200, 0x00000000);
}
u32 addr = nvkm_rd32(device, 0x124120 + (i * 0x0400));
u32 data = nvkm_rd32(device, 0x124124 + (i * 0x0400));
u32 stat = nvkm_rd32(device, 0x124128 + (i * 0x0400));
- nvkm_error(ibus, "ROP%d: %06x %08x (%08x)\n", i, addr, data, stat);
+ nvkm_debug(ibus, "ROP%d: %06x %08x (%08x)\n", i, addr, data, stat);
nvkm_mask(device, 0x124128 + (i * 0x0400), 0x00000200, 0x00000000);
}
u32 addr = nvkm_rd32(device, 0x128120 + (i * 0x0400));
u32 data = nvkm_rd32(device, 0x128124 + (i * 0x0400));
u32 stat = nvkm_rd32(device, 0x128128 + (i * 0x0400));
- nvkm_error(ibus, "GPC%d: %06x %08x (%08x)\n", i, addr, data, stat);
+ nvkm_debug(ibus, "GPC%d: %06x %08x (%08x)\n", i, addr, data, stat);
nvkm_mask(device, 0x128128 + (i * 0x0400), 0x00000200, 0x00000000);
}
u32 addr = nvkm_rd32(device, 0x122120 + (i * 0x0800));
u32 data = nvkm_rd32(device, 0x122124 + (i * 0x0800));
u32 stat = nvkm_rd32(device, 0x122128 + (i * 0x0800));
- nvkm_error(ibus, "HUB%d: %06x %08x (%08x)\n", i, addr, data, stat);
+ nvkm_debug(ibus, "HUB%d: %06x %08x (%08x)\n", i, addr, data, stat);
nvkm_mask(device, 0x122128 + (i * 0x0800), 0x00000200, 0x00000000);
}
u32 addr = nvkm_rd32(device, 0x124120 + (i * 0x0800));
u32 data = nvkm_rd32(device, 0x124124 + (i * 0x0800));
u32 stat = nvkm_rd32(device, 0x124128 + (i * 0x0800));
- nvkm_error(ibus, "ROP%d: %06x %08x (%08x)\n", i, addr, data, stat);
+ nvkm_debug(ibus, "ROP%d: %06x %08x (%08x)\n", i, addr, data, stat);
nvkm_mask(device, 0x124128 + (i * 0x0800), 0x00000200, 0x00000000);
}
u32 addr = nvkm_rd32(device, 0x128120 + (i * 0x0800));
u32 data = nvkm_rd32(device, 0x128124 + (i * 0x0800));
u32 stat = nvkm_rd32(device, 0x128128 + (i * 0x0800));
- nvkm_error(ibus, "GPC%d: %06x %08x (%08x)\n", i, addr, data, stat);
+ nvkm_debug(ibus, "GPC%d: %06x %08x (%08x)\n", i, addr, data, stat);
nvkm_mask(device, 0x128128 + (i * 0x0800), 0x00000200, 0x00000000);
}
*/
#include "priv.h"
+#include <core/msgqueue.h>
#include <subdev/timer.h>
void
);
/* Reset. */
- pmu->func->reset(pmu);
+ if (pmu->func->reset)
+ pmu->func->reset(pmu);
/* Wait for IMEM/DMEM scrubbing to be complete. */
nvkm_msec(device, 2000,
return ret;
}
+static int
+nvkm_pmu_oneinit(struct nvkm_subdev *subdev)
+{
+ struct nvkm_pmu *pmu = nvkm_pmu(subdev);
+ return nvkm_falcon_v1_new(&pmu->subdev, "PMU", 0x10a000, &pmu->falcon);
+}
+
static void *
nvkm_pmu_dtor(struct nvkm_subdev *subdev)
{
struct nvkm_pmu *pmu = nvkm_pmu(subdev);
+ nvkm_msgqueue_del(&pmu->queue);
nvkm_falcon_del(&pmu->falcon);
return nvkm_pmu(subdev);
}
nvkm_pmu = {
.dtor = nvkm_pmu_dtor,
.preinit = nvkm_pmu_preinit,
+ .oneinit = nvkm_pmu_oneinit,
.init = nvkm_pmu_init,
.fini = nvkm_pmu_fini,
.intr = nvkm_pmu_intr,
pmu->func = func;
INIT_WORK(&pmu->recv.work, nvkm_pmu_recv);
init_waitqueue_head(&pmu->recv.wait);
- return nvkm_falcon_v1_new(&pmu->subdev, "PMU", 0x10a000, &pmu->falcon);
+ return 0;
}
int
* DEALINGS IN THE SOFTWARE.
*/
+#include <engine/falcon.h>
+#include <core/msgqueue.h>
#include "priv.h"
+static void
+gm20b_pmu_recv(struct nvkm_pmu *pmu)
+{
+ if (!pmu->queue) {
+ nvkm_warn(&pmu->subdev,
+ "recv function called while no firmware set!\n");
+ return;
+ }
+
+ nvkm_msgqueue_recv(pmu->queue);
+}
+
static const struct nvkm_pmu_func
gm20b_pmu = {
- .reset = gt215_pmu_reset,
+ .intr = gt215_pmu_intr,
+ .recv = gm20b_pmu_recv,
};
int
gm20b_pmu_new(struct nvkm_device *device, int index, struct nvkm_pmu **ppmu)
{
- return nvkm_pmu_new_(&gm20b_pmu, device, index, ppmu);
+ int ret;
+
+ ret = nvkm_pmu_new_(&gm20b_pmu, device, index, ppmu);
+ if (ret)
+ return ret;
+
+ return 0;
}
nvkm-y += nvkm/subdev/secboot/base.o
+nvkm-y += nvkm/subdev/secboot/hs_ucode.o
nvkm-y += nvkm/subdev/secboot/ls_ucode_gr.o
+nvkm-y += nvkm/subdev/secboot/ls_ucode_msgqueue.o
nvkm-y += nvkm/subdev/secboot/acr.o
nvkm-y += nvkm/subdev/secboot/acr_r352.o
nvkm-y += nvkm/subdev/secboot/acr_r361.o
+nvkm-y += nvkm/subdev/secboot/acr_r364.o
+nvkm-y += nvkm/subdev/secboot/acr_r367.o
+nvkm-y += nvkm/subdev/secboot/acr_r375.o
nvkm-y += nvkm/subdev/secboot/gm200.o
nvkm-y += nvkm/subdev/secboot/gm20b.o
+nvkm-y += nvkm/subdev/secboot/gp102.o
void (*dtor)(struct nvkm_acr *);
int (*oneinit)(struct nvkm_acr *, struct nvkm_secboot *);
int (*fini)(struct nvkm_acr *, struct nvkm_secboot *, bool);
- int (*load)(struct nvkm_acr *, struct nvkm_secboot *,
+ int (*load)(struct nvkm_acr *, struct nvkm_falcon *,
struct nvkm_gpuobj *, u64);
int (*reset)(struct nvkm_acr *, struct nvkm_secboot *,
enum nvkm_secboot_falcon);
- int (*start)(struct nvkm_acr *, struct nvkm_secboot *,
- enum nvkm_secboot_falcon);
};
/**
*
* @boot_falcon: ID of the falcon that will perform secure boot
* @managed_falcons: bitfield of falcons managed by this ACR
- * @start_address: virtual start address of the HS bootloader
+ * @optional_falcons: bitfield of falcons we can live without
*/
struct nvkm_acr {
const struct nvkm_acr_func *func;
enum nvkm_secboot_falcon boot_falcon;
unsigned long managed_falcons;
- u32 start_address;
+ unsigned long optional_falcons;
};
void *nvkm_acr_load_firmware(const struct nvkm_subdev *, const char *, size_t);
struct nvkm_acr *acr_r352_new(unsigned long);
struct nvkm_acr *acr_r361_new(unsigned long);
+struct nvkm_acr *acr_r364_new(unsigned long);
+struct nvkm_acr *acr_r367_new(enum nvkm_secboot_falcon, unsigned long);
+struct nvkm_acr *acr_r375_new(enum nvkm_secboot_falcon, unsigned long);
#endif
*/
#include "acr_r352.h"
+#include "hs_ucode.h"
#include <core/gpuobj.h>
#include <core/firmware.h>
#include <engine/falcon.h>
-
-/**
- * struct hsf_fw_header - HS firmware descriptor
- * @sig_dbg_offset: offset of the debug signature
- * @sig_dbg_size: size of the debug signature
- * @sig_prod_offset: offset of the production signature
- * @sig_prod_size: size of the production signature
- * @patch_loc: offset of the offset (sic) of where the signature is
- * @patch_sig: offset of the offset (sic) to add to sig_*_offset
- * @hdr_offset: offset of the load header (see struct hs_load_header)
- * @hdr_size: size of above header
- *
- * This structure is embedded in the HS firmware image at
- * hs_bin_hdr.header_offset.
- */
-struct hsf_fw_header {
- u32 sig_dbg_offset;
- u32 sig_dbg_size;
- u32 sig_prod_offset;
- u32 sig_prod_size;
- u32 patch_loc;
- u32 patch_sig;
- u32 hdr_offset;
- u32 hdr_size;
-};
+#include <subdev/mc.h>
+#include <subdev/timer.h>
+#include <subdev/pmu.h>
+#include <core/msgqueue.h>
+#include <engine/sec2.h>
/**
* struct acr_r352_flcn_bl_desc - DMEM bootloader descriptor
*/
static void
acr_r352_generate_flcn_bl_desc(const struct nvkm_acr *acr,
- const struct ls_ucode_img *_img, u64 wpr_addr,
+ const struct ls_ucode_img *img, u64 wpr_addr,
void *_desc)
{
- struct ls_ucode_img_r352 *img = ls_ucode_img_r352(_img);
struct acr_r352_flcn_bl_desc *desc = _desc;
- const struct ls_ucode_img_desc *pdesc = &_img->ucode_desc;
+ const struct ls_ucode_img_desc *pdesc = &img->ucode_desc;
u64 base, addr_code, addr_data;
- base = wpr_addr + img->lsb_header.ucode_off + pdesc->app_start_offset;
+ base = wpr_addr + img->ucode_off + pdesc->app_start_offset;
addr_code = (base + pdesc->app_resident_code_offset) >> 8;
addr_data = (base + pdesc->app_resident_data_offset) >> 8;
* Low-secure blob creation
*/
+/**
+ * struct acr_r352_lsf_lsb_header - LS firmware header
+ * @signature: signature to verify the firmware against
+ * @ucode_off: offset of the ucode blob in the WPR region. The ucode
+ * blob contains the bootloader, code and data of the
+ * LS falcon
+ * @ucode_size: size of the ucode blob, including bootloader
+ * @data_size: size of the ucode blob data
+ * @bl_code_size: size of the bootloader code
+ * @bl_imem_off: offset in imem of the bootloader
+ * @bl_data_off: offset of the bootloader data in WPR region
+ * @bl_data_size: size of the bootloader data
+ * @app_code_off: offset of the app code relative to ucode_off
+ * @app_code_size: size of the app code
+ * @app_data_off: offset of the app data relative to ucode_off
+ * @app_data_size: size of the app data
+ * @flags: flags for the secure bootloader
+ *
+ * This structure is written into the WPR region for each managed falcon. Each
+ * instance is referenced by the lsb_offset member of the corresponding
+ * lsf_wpr_header.
+ */
+struct acr_r352_lsf_lsb_header {
+ /**
+ * LS falcon signatures
+ * @prd_keys: signature to use in production mode
+ * @dgb_keys: signature to use in debug mode
+ * @b_prd_present: whether the production key is present
+ * @b_dgb_present: whether the debug key is present
+ * @falcon_id: ID of the falcon the ucode applies to
+ */
+ struct {
+ u8 prd_keys[2][16];
+ u8 dbg_keys[2][16];
+ u32 b_prd_present;
+ u32 b_dbg_present;
+ u32 falcon_id;
+ } signature;
+ u32 ucode_off;
+ u32 ucode_size;
+ u32 data_size;
+ u32 bl_code_size;
+ u32 bl_imem_off;
+ u32 bl_data_off;
+ u32 bl_data_size;
+ u32 app_code_off;
+ u32 app_code_size;
+ u32 app_data_off;
+ u32 app_data_size;
+ u32 flags;
+};
+
+/**
+ * struct acr_r352_lsf_wpr_header - LS blob WPR Header
+ * @falcon_id: LS falcon ID
+ * @lsb_offset: offset of the lsb_lsf_header in the WPR region
+ * @bootstrap_owner: secure falcon reponsible for bootstrapping the LS falcon
+ * @lazy_bootstrap: skip bootstrapping by ACR
+ * @status: bootstrapping status
+ *
+ * An array of these is written at the beginning of the WPR region, one for
+ * each managed falcon. The array is terminated by an instance which falcon_id
+ * is LSF_FALCON_ID_INVALID.
+ */
+struct acr_r352_lsf_wpr_header {
+ u32 falcon_id;
+ u32 lsb_offset;
+ u32 bootstrap_owner;
+ u32 lazy_bootstrap;
+ u32 status;
+#define LSF_IMAGE_STATUS_NONE 0
+#define LSF_IMAGE_STATUS_COPY 1
+#define LSF_IMAGE_STATUS_VALIDATION_CODE_FAILED 2
+#define LSF_IMAGE_STATUS_VALIDATION_DATA_FAILED 3
+#define LSF_IMAGE_STATUS_VALIDATION_DONE 4
+#define LSF_IMAGE_STATUS_VALIDATION_SKIPPED 5
+#define LSF_IMAGE_STATUS_BOOTSTRAP_READY 6
+};
+
+/**
+ * struct ls_ucode_img_r352 - ucode image augmented with r352 headers
+ */
+struct ls_ucode_img_r352 {
+ struct ls_ucode_img base;
+
+ struct acr_r352_lsf_wpr_header wpr_header;
+ struct acr_r352_lsf_lsb_header lsb_header;
+};
+#define ls_ucode_img_r352(i) container_of(i, struct ls_ucode_img_r352, base)
+
/**
* ls_ucode_img_load() - create a lsf_ucode_img and load it
*/
* image size
*/
offset = ALIGN(offset, LSF_UCODE_DATA_ALIGN);
- lhdr->ucode_off = offset;
+ _img->ucode_off = lhdr->ucode_off = offset;
offset += _img->ucode_size;
/*
*/
int
acr_r352_ls_write_wpr(struct acr_r352 *acr, struct list_head *imgs,
- struct nvkm_gpuobj *wpr_blob, u32 wpr_addr)
+ struct nvkm_gpuobj *wpr_blob, u64 wpr_addr)
{
struct ls_ucode_img *_img;
u32 pos = 0;
return 0;
}
-/* Both size and address of WPR need to be 128K-aligned */
-#define WPR_ALIGNMENT 0x20000
+/* Both size and address of WPR need to be 256K-aligned */
+#define WPR_ALIGNMENT 0x40000
/**
* acr_r352_prepare_ls_blob() - prepare the LS blob
*
struct ls_ucode_img *img, *t;
unsigned long managed_falcons = acr->base.managed_falcons;
int managed_count = 0;
- u32 image_wpr_size;
+ u32 image_wpr_size, ls_blob_size;
int falcon_id;
int ret;
img = acr->func->ls_ucode_img_load(acr, falcon_id);
if (IS_ERR(img)) {
+ if (acr->base.optional_falcons & BIT(falcon_id)) {
+ managed_falcons &= ~BIT(falcon_id);
+ nvkm_info(subdev, "skipping %s falcon...\n",
+ nvkm_secboot_falcon_name[falcon_id]);
+ continue;
+ }
ret = PTR_ERR(img);
goto cleanup;
}
managed_count++;
}
+ /* Commit the actual list of falcons we will manage from now on */
+ acr->base.managed_falcons = managed_falcons;
+
+ /*
+ * If the boot falcon has a firmare, let it manage the bootstrap of other
+ * falcons.
+ */
+ if (acr->func->ls_func[acr->base.boot_falcon] &&
+ (managed_falcons & BIT(acr->base.boot_falcon))) {
+ for_each_set_bit(falcon_id, &managed_falcons,
+ NVKM_SECBOOT_FALCON_END) {
+ if (falcon_id == acr->base.boot_falcon)
+ continue;
+
+ acr->lazy_bootstrap |= BIT(falcon_id);
+ }
+ }
+
/*
* Fill the WPR and LSF headers with the right offsets and compute
* required WPR size
image_wpr_size = acr->func->ls_fill_headers(acr, &imgs);
image_wpr_size = ALIGN(image_wpr_size, WPR_ALIGNMENT);
+ ls_blob_size = image_wpr_size;
+
+ /*
+ * If we need a shadow area, allocate twice the size and use the
+ * upper half as WPR
+ */
+ if (wpr_size == 0 && acr->func->shadow_blob)
+ ls_blob_size *= 2;
+
/* Allocate GPU object that will contain the WPR region */
- ret = nvkm_gpuobj_new(subdev->device, image_wpr_size, WPR_ALIGNMENT,
+ ret = nvkm_gpuobj_new(subdev->device, ls_blob_size, WPR_ALIGNMENT,
false, NULL, &acr->ls_blob);
if (ret)
goto cleanup;
/* If WPR address and size are not fixed, set them to fit the LS blob */
if (wpr_size == 0) {
wpr_addr = acr->ls_blob->addr;
+ if (acr->func->shadow_blob)
+ wpr_addr += acr->ls_blob->size / 2;
+
wpr_size = image_wpr_size;
/*
* But if the WPR region is set by the bootloader, it is illegal for
-/**
- * acr_r352_hsf_patch_signature() - patch HS blob with correct signature
- */
-static void
-acr_r352_hsf_patch_signature(struct nvkm_secboot *sb, void *acr_image)
-{
- struct fw_bin_header *hsbin_hdr = acr_image;
- struct hsf_fw_header *fw_hdr = acr_image + hsbin_hdr->header_offset;
- void *hs_data = acr_image + hsbin_hdr->data_offset;
- void *sig;
- u32 sig_size;
-
- /* Falcon in debug or production mode? */
- if (sb->boot_falcon->debug) {
- sig = acr_image + fw_hdr->sig_dbg_offset;
- sig_size = fw_hdr->sig_dbg_size;
- } else {
- sig = acr_image + fw_hdr->sig_prod_offset;
- sig_size = fw_hdr->sig_prod_size;
- }
-
- /* Patch signature */
- memcpy(hs_data + fw_hdr->patch_loc, sig + fw_hdr->patch_sig, sig_size);
-}
-
-static void
+void
acr_r352_fixup_hs_desc(struct acr_r352 *acr, struct nvkm_secboot *sb,
- struct hsflcn_acr_desc *desc)
+ void *_desc)
{
+ struct hsflcn_acr_desc *desc = _desc;
struct nvkm_gpuobj *ls_blob = acr->ls_blob;
/* WPR region information if WPR is not fixed */
if (sb->wpr_size == 0) {
- u32 wpr_start = ls_blob->addr;
- u32 wpr_end = wpr_start + ls_blob->size;
+ u64 wpr_start = ls_blob->addr;
+ u64 wpr_end = wpr_start + ls_blob->size;
desc->wpr_region_id = 1;
desc->regions.no_regions = 2;
bl_desc->code_dma_base = lower_32_bits(addr_code);
bl_desc->non_sec_code_off = hdr->non_sec_code_off;
bl_desc->non_sec_code_size = hdr->non_sec_code_size;
- bl_desc->sec_code_off = hdr->app[0].sec_code_off;
- bl_desc->sec_code_size = hdr->app[0].sec_code_size;
+ bl_desc->sec_code_off = hsf_load_header_app_off(hdr, 0);
+ bl_desc->sec_code_size = hsf_load_header_app_size(hdr, 0);
bl_desc->code_entry_point = 0;
bl_desc->data_dma_base = lower_32_bits(addr_data);
bl_desc->data_size = hdr->data_size;
void *acr_data;
int ret;
- acr_image = nvkm_acr_load_firmware(subdev, fw, 0);
+ acr_image = hs_ucode_load_blob(subdev, sb->boot_falcon, fw);
if (IS_ERR(acr_image))
return PTR_ERR(acr_image);
load_hdr = acr_image + fw_hdr->hdr_offset;
acr_data = acr_image + hsbin_hdr->data_offset;
- /* Patch signature */
- acr_r352_hsf_patch_signature(sb, acr_image);
-
/* Patch descriptor with WPR information? */
if (patch) {
struct hsflcn_acr_desc *desc;
desc = acr_data + load_hdr->data_dma_base;
- acr_r352_fixup_hs_desc(acr, sb, desc);
+ acr->func->fixup_hs_desc(acr, sb, desc);
}
if (load_hdr->num_apps > ACR_R352_MAX_APPS) {
goto cleanup;
}
memcpy(load_header, load_hdr, sizeof(*load_header) +
- (sizeof(load_hdr->app[0]) * load_hdr->num_apps));
+ (sizeof(load_hdr->apps[0]) * 2 * load_hdr->num_apps));
/* Create ACR blob and copy HS data to it */
ret = nvkm_gpuobj_new(subdev->device, ALIGN(hsbin_hdr->data_size, 256),
return ret;
}
-static int
-acr_r352_prepare_hsbl_blob(struct acr_r352 *acr)
-{
- const struct nvkm_subdev *subdev = acr->base.subdev;
- struct fw_bin_header *hdr;
- struct fw_bl_desc *hsbl_desc;
-
- acr->hsbl_blob = nvkm_acr_load_firmware(subdev, "acr/bl", 0);
- if (IS_ERR(acr->hsbl_blob)) {
- int ret = PTR_ERR(acr->hsbl_blob);
-
- acr->hsbl_blob = NULL;
- return ret;
- }
-
- hdr = acr->hsbl_blob;
- hsbl_desc = acr->hsbl_blob + hdr->header_offset;
-
- /* virtual start address for boot vector */
- acr->base.start_address = hsbl_desc->start_tag << 8;
-
- return 0;
-}
-
/**
* acr_r352_load_blobs - load blobs common to all ACR V1 versions.
*
int
acr_r352_load_blobs(struct acr_r352 *acr, struct nvkm_secboot *sb)
{
+ struct nvkm_subdev *subdev = &sb->subdev;
int ret;
/* Firmware already loaded? */
/* Load the HS firmware bootloader */
if (!acr->hsbl_blob) {
- ret = acr_r352_prepare_hsbl_blob(acr);
- if (ret)
+ acr->hsbl_blob = nvkm_acr_load_firmware(subdev, "acr/bl", 0);
+ if (IS_ERR(acr->hsbl_blob)) {
+ ret = PTR_ERR(acr->hsbl_blob);
+ acr->hsbl_blob = NULL;
return ret;
+ }
+
+ if (acr->base.boot_falcon != NVKM_SECBOOT_FALCON_PMU) {
+ acr->hsbl_unload_blob = nvkm_acr_load_firmware(subdev,
+ "acr/unload_bl", 0);
+ if (IS_ERR(acr->hsbl_unload_blob)) {
+ ret = PTR_ERR(acr->hsbl_unload_blob);
+ acr->hsbl_unload_blob = NULL;
+ return ret;
+ }
+ } else {
+ acr->hsbl_unload_blob = acr->hsbl_blob;
+ }
}
acr->firmware_ok = true;
}
/**
- * acr_r352_load() - prepare HS falcon to run the specified blob, mapped
- * at GPU address offset.
+ * acr_r352_load() - prepare HS falcon to run the specified blob, mapped.
+ *
+ * Returns the start address to use, or a negative error value.
*/
static int
-acr_r352_load(struct nvkm_acr *_acr, struct nvkm_secboot *sb,
+acr_r352_load(struct nvkm_acr *_acr, struct nvkm_falcon *falcon,
struct nvkm_gpuobj *blob, u64 offset)
{
struct acr_r352 *acr = acr_r352(_acr);
- struct nvkm_falcon *falcon = sb->boot_falcon;
- struct fw_bin_header *hdr = acr->hsbl_blob;
- struct fw_bl_desc *hsbl_desc = acr->hsbl_blob + hdr->header_offset;
- void *blob_data = acr->hsbl_blob + hdr->data_offset;
- void *hsbl_code = blob_data + hsbl_desc->code_off;
- void *hsbl_data = blob_data + hsbl_desc->data_off;
- u32 code_size = ALIGN(hsbl_desc->code_size, 256);
- const struct hsf_load_header *load_hdr;
const u32 bl_desc_size = acr->func->hs_bl_desc_size;
+ const struct hsf_load_header *load_hdr;
+ struct fw_bin_header *bl_hdr;
+ struct fw_bl_desc *hsbl_desc;
+ void *bl, *blob_data, *hsbl_code, *hsbl_data;
+ u32 code_size;
u8 bl_desc[bl_desc_size];
/* Find the bootloader descriptor for our blob and copy it */
if (blob == acr->load_blob) {
load_hdr = &acr->load_bl_header;
+ bl = acr->hsbl_blob;
} else if (blob == acr->unload_blob) {
load_hdr = &acr->unload_bl_header;
+ bl = acr->hsbl_unload_blob;
} else {
nvkm_error(_acr->subdev, "invalid secure boot blob!\n");
return -EINVAL;
}
+ bl_hdr = bl;
+ hsbl_desc = bl + bl_hdr->header_offset;
+ blob_data = bl + bl_hdr->data_offset;
+ hsbl_code = blob_data + hsbl_desc->code_off;
+ hsbl_data = blob_data + hsbl_desc->data_off;
+ code_size = ALIGN(hsbl_desc->code_size, 256);
+
/*
* Copy HS bootloader data
*/
nvkm_falcon_load_dmem(falcon, bl_desc, hsbl_desc->dmem_load_off,
bl_desc_size, 0);
- return 0;
+ return hsbl_desc->start_tag << 8;
}
static int
acr_r352_shutdown(struct acr_r352 *acr, struct nvkm_secboot *sb)
{
+ struct nvkm_subdev *subdev = &sb->subdev;
int i;
/* Run the unload blob to unprotect the WPR region */
if (acr->unload_blob && sb->wpr_set) {
int ret;
- nvkm_debug(&sb->subdev, "running HS unload blob\n");
- ret = sb->func->run_blob(sb, acr->unload_blob);
- if (ret)
+ nvkm_debug(subdev, "running HS unload blob\n");
+ ret = sb->func->run_blob(sb, acr->unload_blob, sb->halt_falcon);
+ if (ret < 0)
return ret;
- nvkm_debug(&sb->subdev, "HS unload blob completed\n");
+ /*
+ * Unload blob will return this error code - it is not an error
+ * and the expected behavior on RM as well
+ */
+ if (ret && ret != 0x1d) {
+ nvkm_error(subdev, "HS unload failed, ret 0x%08x", ret);
+ return -EINVAL;
+ }
+ nvkm_debug(subdev, "HS unload blob completed\n");
}
for (i = 0; i < NVKM_SECBOOT_FALCON_END; i++)
return 0;
}
+/**
+ * Check if the WPR region has been indeed set by the ACR firmware, and
+ * matches where it should be.
+ */
+static bool
+acr_r352_wpr_is_set(const struct acr_r352 *acr, const struct nvkm_secboot *sb)
+{
+ const struct nvkm_subdev *subdev = &sb->subdev;
+ const struct nvkm_device *device = subdev->device;
+ u64 wpr_lo, wpr_hi;
+ u64 wpr_range_lo, wpr_range_hi;
+
+ nvkm_wr32(device, 0x100cd4, 0x2);
+ wpr_lo = (nvkm_rd32(device, 0x100cd4) & ~0xff);
+ wpr_lo <<= 8;
+ nvkm_wr32(device, 0x100cd4, 0x3);
+ wpr_hi = (nvkm_rd32(device, 0x100cd4) & ~0xff);
+ wpr_hi <<= 8;
+
+ if (sb->wpr_size != 0) {
+ wpr_range_lo = sb->wpr_addr;
+ wpr_range_hi = wpr_range_lo + sb->wpr_size;
+ } else {
+ wpr_range_lo = acr->ls_blob->addr;
+ wpr_range_hi = wpr_range_lo + acr->ls_blob->size;
+ }
+
+ return (wpr_lo >= wpr_range_lo && wpr_lo < wpr_range_hi &&
+ wpr_hi > wpr_range_lo && wpr_hi <= wpr_range_hi);
+}
+
static int
acr_r352_bootstrap(struct acr_r352 *acr, struct nvkm_secboot *sb)
{
+ const struct nvkm_subdev *subdev = &sb->subdev;
+ unsigned long managed_falcons = acr->base.managed_falcons;
+ u32 reg;
+ int falcon_id;
int ret;
if (sb->wpr_set)
if (ret)
return ret;
- nvkm_debug(&sb->subdev, "running HS load blob\n");
- ret = sb->func->run_blob(sb, acr->load_blob);
+ nvkm_debug(subdev, "running HS load blob\n");
+ ret = sb->func->run_blob(sb, acr->load_blob, sb->boot_falcon);
/* clear halt interrupt */
nvkm_falcon_clear_interrupt(sb->boot_falcon, 0x10);
- if (ret)
+ sb->wpr_set = acr_r352_wpr_is_set(acr, sb);
+ if (ret < 0) {
return ret;
- nvkm_debug(&sb->subdev, "HS load blob completed\n");
+ } else if (ret > 0) {
+ nvkm_error(subdev, "HS load failed, ret 0x%08x", ret);
+ return -EINVAL;
+ }
+ nvkm_debug(subdev, "HS load blob completed\n");
+ /* WPR must be set at this point */
+ if (!sb->wpr_set) {
+ nvkm_error(subdev, "ACR blob completed but WPR not set!\n");
+ return -EINVAL;
+ }
+
+ /* Run LS firmwares post_run hooks */
+ for_each_set_bit(falcon_id, &managed_falcons, NVKM_SECBOOT_FALCON_END) {
+ const struct acr_r352_ls_func *func =
+ acr->func->ls_func[falcon_id];
+
+ if (func->post_run)
+ func->post_run(&acr->base, sb);
+ }
+
+ /* Re-start ourselves if we are managed */
+ if (!nvkm_secboot_is_managed(sb, acr->base.boot_falcon))
+ return 0;
+
+ /* Enable interrupts */
+ nvkm_falcon_wr32(sb->boot_falcon, 0x10, 0xff);
+ nvkm_mc_intr_mask(subdev->device, sb->boot_falcon->owner->index, true);
+
+ /* Start LS firmware on boot falcon */
+ nvkm_falcon_start(sb->boot_falcon);
+
+ /*
+ * There is a bug where the LS firmware sometimes require to be started
+ * twice (this happens only on SEC). Detect and workaround that
+ * condition.
+ *
+ * Once started, the falcon will end up in STOPPED condition (bit 5)
+ * if successful, or in HALT condition (bit 4) if not.
+ */
+ nvkm_msec(subdev->device, 1,
+ if ((reg = nvkm_rd32(subdev->device,
+ sb->boot_falcon->addr + 0x100)
+ & 0x30) != 0)
+ break;
+ );
+ if (reg & BIT(4)) {
+ nvkm_debug(subdev, "applying workaround for start bug...");
+ nvkm_falcon_start(sb->boot_falcon);
+ nvkm_msec(subdev->device, 1,
+ if ((reg = nvkm_rd32(subdev->device,
+ sb->boot_falcon->addr + 0x100)
+ & 0x30) != 0)
+ break;
+ );
+ if (reg & BIT(4)) {
+ nvkm_error(subdev, "%s failed to start\n",
+ nvkm_secboot_falcon_name[acr->base.boot_falcon]);
+ return -EINVAL;
+ }
+ }
- sb->wpr_set = true;
+ nvkm_debug(subdev, "%s started\n",
+ nvkm_secboot_falcon_name[acr->base.boot_falcon]);
return 0;
}
-/*
- * acr_r352_reset() - execute secure boot from the prepared state
+/**
+ * acr_r352_reset_nopmu - dummy reset method when no PMU firmware is loaded
*
- * Load the HS bootloader and ask the falcon to run it. This will in turn
- * load the HS firmware and run it, so once the falcon stops all the managed
- * falcons should have their LS firmware loaded and be ready to run.
+ * Reset is done by re-executing secure boot from scratch, with lazy bootstrap
+ * disabled. This has the effect of making all managed falcons ready-to-run.
*/
static int
-acr_r352_reset(struct nvkm_acr *_acr, struct nvkm_secboot *sb,
- enum nvkm_secboot_falcon falcon)
+acr_r352_reset_nopmu(struct acr_r352 *acr, struct nvkm_secboot *sb,
+ enum nvkm_secboot_falcon falcon)
{
- struct acr_r352 *acr = acr_r352(_acr);
int ret;
/*
- * Dummy GM200 implementation: perform secure boot each time we are
- * called on FECS. Since only FECS and GPCCS are managed and started
- * together, this ought to be safe.
- *
- * Once we have proper PMU firmware and support, this will be changed
- * to a proper call to the PMU method.
+ * Perform secure boot each time we are called on FECS. Since only FECS
+ * and GPCCS are managed and started together, this ought to be safe.
*/
if (falcon != NVKM_SECBOOT_FALCON_FECS)
goto end;
if (ret)
return ret;
- acr_r352_bootstrap(acr, sb);
+ ret = acr_r352_bootstrap(acr, sb);
if (ret)
return ret;
return 0;
}
+/*
+ * acr_r352_reset() - execute secure boot from the prepared state
+ *
+ * Load the HS bootloader and ask the falcon to run it. This will in turn
+ * load the HS firmware and run it, so once the falcon stops all the managed
+ * falcons should have their LS firmware loaded and be ready to run.
+ */
static int
-acr_r352_start(struct nvkm_acr *_acr, struct nvkm_secboot *sb,
- enum nvkm_secboot_falcon falcon)
+acr_r352_reset(struct nvkm_acr *_acr, struct nvkm_secboot *sb,
+ enum nvkm_secboot_falcon falcon)
{
struct acr_r352 *acr = acr_r352(_acr);
- const struct nvkm_subdev *subdev = &sb->subdev;
- int base;
+ struct nvkm_msgqueue *queue;
+ const char *fname = nvkm_secboot_falcon_name[falcon];
+ bool wpr_already_set = sb->wpr_set;
+ int ret;
- switch (falcon) {
- case NVKM_SECBOOT_FALCON_FECS:
- base = 0x409000;
+ /* Make sure secure boot is performed */
+ ret = acr_r352_bootstrap(acr, sb);
+ if (ret)
+ return ret;
+
+ /* No PMU interface? */
+ if (!nvkm_secboot_is_managed(sb, _acr->boot_falcon)) {
+ /* Redo secure boot entirely if it was already done */
+ if (wpr_already_set)
+ return acr_r352_reset_nopmu(acr, sb, falcon);
+ /* Else return the result of the initial invokation */
+ else
+ return ret;
+ }
+
+ switch (_acr->boot_falcon) {
+ case NVKM_SECBOOT_FALCON_PMU:
+ queue = sb->subdev.device->pmu->queue;
break;
- case NVKM_SECBOOT_FALCON_GPCCS:
- base = 0x41a000;
+ case NVKM_SECBOOT_FALCON_SEC2:
+ queue = sb->subdev.device->sec2->queue;
break;
default:
- nvkm_error(subdev, "cannot start unhandled falcon!\n");
return -EINVAL;
}
- nvkm_wr32(subdev->device, base + 0x130, 0x00000002);
- acr->falcon_state[falcon] = RUNNING;
+ /* Otherwise just ask the LS firmware to reset the falcon */
+ nvkm_debug(&sb->subdev, "resetting %s falcon\n", fname);
+ ret = nvkm_msgqueue_acr_boot_falcon(queue, falcon);
+ if (ret) {
+ nvkm_error(&sb->subdev, "cannot boot %s falcon\n", fname);
+ return ret;
+ }
+ nvkm_debug(&sb->subdev, "falcon %s reset\n", fname);
return 0;
}
nvkm_gpuobj_del(&acr->unload_blob);
+ if (_acr->boot_falcon != NVKM_SECBOOT_FALCON_PMU)
+ kfree(acr->hsbl_unload_blob);
kfree(acr->hsbl_blob);
nvkm_gpuobj_del(&acr->load_blob);
nvkm_gpuobj_del(&acr->ls_blob);
.lhdr_flags = LSF_FLAG_FORCE_PRIV_LOAD,
};
+
+
+/**
+ * struct acr_r352_pmu_bl_desc - PMU DMEM bootloader descriptor
+ * @dma_idx: DMA context to be used by BL while loading code/data
+ * @code_dma_base: 256B-aligned Physical FB Address where code is located
+ * @total_code_size: total size of the code part in the ucode
+ * @code_size_to_load: size of the code part to load in PMU IMEM.
+ * @code_entry_point: entry point in the code.
+ * @data_dma_base: Physical FB address where data part of ucode is located
+ * @data_size: Total size of the data portion.
+ * @overlay_dma_base: Physical Fb address for resident code present in ucode
+ * @argc: Total number of args
+ * @argv: offset where args are copied into PMU's DMEM.
+ *
+ * Structure used by the PMU bootloader to load the rest of the code
+ */
+struct acr_r352_pmu_bl_desc {
+ u32 dma_idx;
+ u32 code_dma_base;
+ u32 code_size_total;
+ u32 code_size_to_load;
+ u32 code_entry_point;
+ u32 data_dma_base;
+ u32 data_size;
+ u32 overlay_dma_base;
+ u32 argc;
+ u32 argv;
+ u16 code_dma_base1;
+ u16 data_dma_base1;
+ u16 overlay_dma_base1;
+};
+
+/**
+ * acr_r352_generate_pmu_bl_desc() - populate a DMEM BL descriptor for PMU LS image
+ *
+ */
+static void
+acr_r352_generate_pmu_bl_desc(const struct nvkm_acr *acr,
+ const struct ls_ucode_img *img, u64 wpr_addr,
+ void *_desc)
+{
+ const struct ls_ucode_img_desc *pdesc = &img->ucode_desc;
+ const struct nvkm_pmu *pmu = acr->subdev->device->pmu;
+ struct acr_r352_pmu_bl_desc *desc = _desc;
+ u64 base;
+ u64 addr_code;
+ u64 addr_data;
+ u32 addr_args;
+
+ base = wpr_addr + img->ucode_off + pdesc->app_start_offset;
+ addr_code = (base + pdesc->app_resident_code_offset) >> 8;
+ addr_data = (base + pdesc->app_resident_data_offset) >> 8;
+ addr_args = pmu->falcon->data.limit;
+ addr_args -= NVKM_MSGQUEUE_CMDLINE_SIZE;
+
+ desc->dma_idx = FALCON_DMAIDX_UCODE;
+ desc->code_dma_base = lower_32_bits(addr_code);
+ desc->code_dma_base1 = upper_32_bits(addr_code);
+ desc->code_size_total = pdesc->app_size;
+ desc->code_size_to_load = pdesc->app_resident_code_size;
+ desc->code_entry_point = pdesc->app_imem_entry;
+ desc->data_dma_base = lower_32_bits(addr_data);
+ desc->data_dma_base1 = upper_32_bits(addr_data);
+ desc->data_size = pdesc->app_resident_data_size;
+ desc->overlay_dma_base = lower_32_bits(addr_code);
+ desc->overlay_dma_base1 = upper_32_bits(addr_code);
+ desc->argc = 1;
+ desc->argv = addr_args;
+}
+
+static const struct acr_r352_ls_func
+acr_r352_ls_pmu_func = {
+ .load = acr_ls_ucode_load_pmu,
+ .generate_bl_desc = acr_r352_generate_pmu_bl_desc,
+ .bl_desc_size = sizeof(struct acr_r352_pmu_bl_desc),
+ .post_run = acr_ls_pmu_post_run,
+};
+
const struct acr_r352_func
acr_r352_func = {
+ .fixup_hs_desc = acr_r352_fixup_hs_desc,
.generate_hs_bl_desc = acr_r352_generate_hs_bl_desc,
.hs_bl_desc_size = sizeof(struct acr_r352_flcn_bl_desc),
.ls_ucode_img_load = acr_r352_ls_ucode_img_load,
.ls_func = {
[NVKM_SECBOOT_FALCON_FECS] = &acr_r352_ls_fecs_func,
[NVKM_SECBOOT_FALCON_GPCCS] = &acr_r352_ls_gpccs_func,
+ [NVKM_SECBOOT_FALCON_PMU] = &acr_r352_ls_pmu_func,
},
};
.fini = acr_r352_fini,
.load = acr_r352_load,
.reset = acr_r352_reset,
- .start = acr_r352_start,
};
struct nvkm_acr *
unsigned long managed_falcons)
{
struct acr_r352 *acr;
+ int i;
+
+ /* Check that all requested falcons are supported */
+ for_each_set_bit(i, &managed_falcons, NVKM_SECBOOT_FALCON_END) {
+ if (!func->ls_func[i])
+ return ERR_PTR(-ENOTSUPP);
+ }
acr = kzalloc(sizeof(*acr), GFP_KERNEL);
if (!acr)
#include "acr.h"
#include "ls_ucode.h"
+#include "hs_ucode.h"
struct ls_ucode_img;
#define ACR_R352_MAX_APPS 8
-/*
- *
- * LS blob structures
- *
- */
-
-/**
- * struct acr_r352_lsf_lsb_header - LS firmware header
- * @signature: signature to verify the firmware against
- * @ucode_off: offset of the ucode blob in the WPR region. The ucode
- * blob contains the bootloader, code and data of the
- * LS falcon
- * @ucode_size: size of the ucode blob, including bootloader
- * @data_size: size of the ucode blob data
- * @bl_code_size: size of the bootloader code
- * @bl_imem_off: offset in imem of the bootloader
- * @bl_data_off: offset of the bootloader data in WPR region
- * @bl_data_size: size of the bootloader data
- * @app_code_off: offset of the app code relative to ucode_off
- * @app_code_size: size of the app code
- * @app_data_off: offset of the app data relative to ucode_off
- * @app_data_size: size of the app data
- * @flags: flags for the secure bootloader
- *
- * This structure is written into the WPR region for each managed falcon. Each
- * instance is referenced by the lsb_offset member of the corresponding
- * lsf_wpr_header.
- */
-struct acr_r352_lsf_lsb_header {
- /**
- * LS falcon signatures
- * @prd_keys: signature to use in production mode
- * @dgb_keys: signature to use in debug mode
- * @b_prd_present: whether the production key is present
- * @b_dgb_present: whether the debug key is present
- * @falcon_id: ID of the falcon the ucode applies to
- */
- struct {
- u8 prd_keys[2][16];
- u8 dbg_keys[2][16];
- u32 b_prd_present;
- u32 b_dbg_present;
- u32 falcon_id;
- } signature;
- u32 ucode_off;
- u32 ucode_size;
- u32 data_size;
- u32 bl_code_size;
- u32 bl_imem_off;
- u32 bl_data_off;
- u32 bl_data_size;
- u32 app_code_off;
- u32 app_code_size;
- u32 app_data_off;
- u32 app_data_size;
- u32 flags;
#define LSF_FLAG_LOAD_CODE_AT_0 1
#define LSF_FLAG_DMACTL_REQ_CTX 4
#define LSF_FLAG_FORCE_PRIV_LOAD 8
-};
-
-/**
- * struct acr_r352_lsf_wpr_header - LS blob WPR Header
- * @falcon_id: LS falcon ID
- * @lsb_offset: offset of the lsb_lsf_header in the WPR region
- * @bootstrap_owner: secure falcon reponsible for bootstrapping the LS falcon
- * @lazy_bootstrap: skip bootstrapping by ACR
- * @status: bootstrapping status
- *
- * An array of these is written at the beginning of the WPR region, one for
- * each managed falcon. The array is terminated by an instance which falcon_id
- * is LSF_FALCON_ID_INVALID.
- */
-struct acr_r352_lsf_wpr_header {
- u32 falcon_id;
- u32 lsb_offset;
- u32 bootstrap_owner;
- u32 lazy_bootstrap;
- u32 status;
-#define LSF_IMAGE_STATUS_NONE 0
-#define LSF_IMAGE_STATUS_COPY 1
-#define LSF_IMAGE_STATUS_VALIDATION_CODE_FAILED 2
-#define LSF_IMAGE_STATUS_VALIDATION_DATA_FAILED 3
-#define LSF_IMAGE_STATUS_VALIDATION_DONE 4
-#define LSF_IMAGE_STATUS_VALIDATION_SKIPPED 5
-#define LSF_IMAGE_STATUS_BOOTSTRAP_READY 6
-};
-
-/**
- * struct ls_ucode_img_r352 - ucode image augmented with r352 headers
- */
-struct ls_ucode_img_r352 {
- struct ls_ucode_img base;
-
- struct acr_r352_lsf_wpr_header wpr_header;
- struct acr_r352_lsf_lsb_header lsb_header;
-};
-#define ls_ucode_img_r352(i) container_of(i, struct ls_ucode_img_r352, base)
-
-
-/*
- * HS blob structures
- */
-struct hsf_load_header_app {
- u32 sec_code_off;
- u32 sec_code_size;
-};
+static inline u32
+hsf_load_header_app_off(const struct hsf_load_header *hdr, u32 app)
+{
+ return hdr->apps[app];
+}
-/**
- * struct hsf_load_header - HS firmware load header
- */
-struct hsf_load_header {
- u32 non_sec_code_off;
- u32 non_sec_code_size;
- u32 data_dma_base;
- u32 data_size;
- u32 num_apps;
- struct hsf_load_header_app app[0];
-};
+static inline u32
+hsf_load_header_app_size(const struct hsf_load_header *hdr, u32 app)
+{
+ return hdr->apps[hdr->num_apps + app];
+}
/**
* struct acr_r352_ls_func - manages a single LS firmware
* @generate_bl_desc: function called on a block of bl_desc_size to generate the
* proper bootloader descriptor for this LS firmware
* @bl_desc_size: size of the bootloader descriptor
+ * @post_run: hook called right after the ACR is executed
* @lhdr_flags: LS flags
*/
struct acr_r352_ls_func {
void (*generate_bl_desc)(const struct nvkm_acr *,
const struct ls_ucode_img *, u64, void *);
u32 bl_desc_size;
+ void (*post_run)(const struct nvkm_acr *, const struct nvkm_secboot *);
u32 lhdr_flags;
};
struct acr_r352_func {
void (*generate_hs_bl_desc)(const struct hsf_load_header *, void *,
u64);
+ void (*fixup_hs_desc)(struct acr_r352 *, struct nvkm_secboot *, void *);
u32 hs_bl_desc_size;
+ bool shadow_blob;
struct ls_ucode_img *(*ls_ucode_img_load)(const struct acr_r352 *,
enum nvkm_secboot_falcon);
int (*ls_fill_headers)(struct acr_r352 *, struct list_head *);
int (*ls_write_wpr)(struct acr_r352 *, struct list_head *,
- struct nvkm_gpuobj *, u32);
+ struct nvkm_gpuobj *, u64);
const struct acr_r352_ls_func *ls_func[NVKM_SECBOOT_FALCON_END];
};
struct nvkm_gpuobj *load_blob;
struct {
struct hsf_load_header load_bl_header;
- struct hsf_load_header_app __load_apps[ACR_R352_MAX_APPS];
+ u32 __load_apps[ACR_R352_MAX_APPS * 2];
};
/* HS FW - unlock WPR region (dGPU only) */
struct nvkm_gpuobj *unload_blob;
struct {
struct hsf_load_header unload_bl_header;
- struct hsf_load_header_app __unload_apps[ACR_R352_MAX_APPS];
+ u32 __unload_apps[ACR_R352_MAX_APPS * 2];
};
/* HS bootloader */
void *hsbl_blob;
+ /* HS bootloader for unload blob, if using a different falcon */
+ void *hsbl_unload_blob;
+
/* LS FWs, to be loaded by the HS ACR */
struct nvkm_gpuobj *ls_blob;
enum nvkm_secboot_falcon);
int acr_r352_ls_fill_headers(struct acr_r352 *, struct list_head *);
int acr_r352_ls_write_wpr(struct acr_r352 *, struct list_head *,
- struct nvkm_gpuobj *, u32);
+ struct nvkm_gpuobj *, u64);
+
+void acr_r352_fixup_hs_desc(struct acr_r352 *, struct nvkm_secboot *, void *);
#endif
* DEALINGS IN THE SOFTWARE.
*/
-#include "acr_r352.h"
+#include "acr_r361.h"
#include <engine/falcon.h>
-
-/**
- * struct acr_r361_flcn_bl_desc - DMEM bootloader descriptor
- * @signature: 16B signature for secure code. 0s if no secure code
- * @ctx_dma: DMA context to be used by BL while loading code/data
- * @code_dma_base: 256B-aligned Physical FB Address where code is located
- * (falcon's $xcbase register)
- * @non_sec_code_off: offset from code_dma_base where the non-secure code is
- * located. The offset must be multiple of 256 to help perf
- * @non_sec_code_size: the size of the nonSecure code part.
- * @sec_code_off: offset from code_dma_base where the secure code is
- * located. The offset must be multiple of 256 to help perf
- * @sec_code_size: offset from code_dma_base where the secure code is
- * located. The offset must be multiple of 256 to help perf
- * @code_entry_point: code entry point which will be invoked by BL after
- * code is loaded.
- * @data_dma_base: 256B aligned Physical FB Address where data is located.
- * (falcon's $xdbase register)
- * @data_size: size of data block. Should be multiple of 256B
- *
- * Structure used by the bootloader to load the rest of the code. This has
- * to be filled by host and copied into DMEM at offset provided in the
- * hsflcn_bl_desc.bl_desc_dmem_load_off.
- */
-struct acr_r361_flcn_bl_desc {
- u32 reserved[4];
- u32 signature[4];
- u32 ctx_dma;
- struct flcn_u64 code_dma_base;
- u32 non_sec_code_off;
- u32 non_sec_code_size;
- u32 sec_code_off;
- u32 sec_code_size;
- u32 code_entry_point;
- struct flcn_u64 data_dma_base;
- u32 data_size;
-};
+#include <core/msgqueue.h>
+#include <subdev/pmu.h>
+#include <engine/sec2.h>
static void
acr_r361_generate_flcn_bl_desc(const struct nvkm_acr *acr,
- const struct ls_ucode_img *_img, u64 wpr_addr,
+ const struct ls_ucode_img *img, u64 wpr_addr,
void *_desc)
{
- struct ls_ucode_img_r352 *img = ls_ucode_img_r352(_img);
struct acr_r361_flcn_bl_desc *desc = _desc;
- const struct ls_ucode_img_desc *pdesc = &img->base.ucode_desc;
+ const struct ls_ucode_img_desc *pdesc = &img->ucode_desc;
u64 base, addr_code, addr_data;
- base = wpr_addr + img->lsb_header.ucode_off + pdesc->app_start_offset;
+ base = wpr_addr + img->ucode_off + pdesc->app_start_offset;
addr_code = base + pdesc->app_resident_code_offset;
addr_data = base + pdesc->app_resident_data_offset;
desc->data_size = pdesc->app_resident_data_size;
}
-static void
+void
acr_r361_generate_hs_bl_desc(const struct hsf_load_header *hdr, void *_bl_desc,
u64 offset)
{
bl_desc->code_dma_base = u64_to_flcn64(offset);
bl_desc->non_sec_code_off = hdr->non_sec_code_off;
bl_desc->non_sec_code_size = hdr->non_sec_code_size;
- bl_desc->sec_code_off = hdr->app[0].sec_code_off;
- bl_desc->sec_code_size = hdr->app[0].sec_code_size;
+ bl_desc->sec_code_off = hsf_load_header_app_off(hdr, 0);
+ bl_desc->sec_code_size = hsf_load_header_app_size(hdr, 0);
bl_desc->code_entry_point = 0;
bl_desc->data_dma_base = u64_to_flcn64(offset + hdr->data_dma_base);
bl_desc->data_size = hdr->data_size;
.lhdr_flags = LSF_FLAG_FORCE_PRIV_LOAD,
};
+struct acr_r361_pmu_bl_desc {
+ u32 reserved;
+ u32 dma_idx;
+ struct flcn_u64 code_dma_base;
+ u32 total_code_size;
+ u32 code_size_to_load;
+ u32 code_entry_point;
+ struct flcn_u64 data_dma_base;
+ u32 data_size;
+ struct flcn_u64 overlay_dma_base;
+ u32 argc;
+ u32 argv;
+};
+
+static void
+acr_r361_generate_pmu_bl_desc(const struct nvkm_acr *acr,
+ const struct ls_ucode_img *img, u64 wpr_addr,
+ void *_desc)
+{
+ const struct ls_ucode_img_desc *pdesc = &img->ucode_desc;
+ const struct nvkm_pmu *pmu = acr->subdev->device->pmu;
+ struct acr_r361_pmu_bl_desc *desc = _desc;
+ u64 base, addr_code, addr_data;
+ u32 addr_args;
+
+ base = wpr_addr + img->ucode_off + pdesc->app_start_offset;
+ addr_code = base + pdesc->app_resident_code_offset;
+ addr_data = base + pdesc->app_resident_data_offset;
+ addr_args = pmu->falcon->data.limit;
+ addr_args -= NVKM_MSGQUEUE_CMDLINE_SIZE;
+
+ desc->dma_idx = FALCON_DMAIDX_UCODE;
+ desc->code_dma_base = u64_to_flcn64(addr_code);
+ desc->total_code_size = pdesc->app_size;
+ desc->code_size_to_load = pdesc->app_resident_code_size;
+ desc->code_entry_point = pdesc->app_imem_entry;
+ desc->data_dma_base = u64_to_flcn64(addr_data);
+ desc->data_size = pdesc->app_resident_data_size;
+ desc->overlay_dma_base = u64_to_flcn64(addr_code);
+ desc->argc = 1;
+ desc->argv = addr_args;
+}
+
+const struct acr_r352_ls_func
+acr_r361_ls_pmu_func = {
+ .load = acr_ls_ucode_load_pmu,
+ .generate_bl_desc = acr_r361_generate_pmu_bl_desc,
+ .bl_desc_size = sizeof(struct acr_r361_pmu_bl_desc),
+ .post_run = acr_ls_pmu_post_run,
+};
+
+static void
+acr_r361_generate_sec2_bl_desc(const struct nvkm_acr *acr,
+ const struct ls_ucode_img *img, u64 wpr_addr,
+ void *_desc)
+{
+ const struct ls_ucode_img_desc *pdesc = &img->ucode_desc;
+ const struct nvkm_sec2 *sec = acr->subdev->device->sec2;
+ struct acr_r361_pmu_bl_desc *desc = _desc;
+ u64 base, addr_code, addr_data;
+ u32 addr_args;
+
+ base = wpr_addr + img->ucode_off + pdesc->app_start_offset;
+ /* For some reason we should not add app_resident_code_offset here */
+ addr_code = base;
+ addr_data = base + pdesc->app_resident_data_offset;
+ addr_args = sec->falcon->data.limit;
+ addr_args -= NVKM_MSGQUEUE_CMDLINE_SIZE;
+
+ desc->dma_idx = FALCON_SEC2_DMAIDX_UCODE;
+ desc->code_dma_base = u64_to_flcn64(addr_code);
+ desc->total_code_size = pdesc->app_size;
+ desc->code_size_to_load = pdesc->app_resident_code_size;
+ desc->code_entry_point = pdesc->app_imem_entry;
+ desc->data_dma_base = u64_to_flcn64(addr_data);
+ desc->data_size = pdesc->app_resident_data_size;
+ desc->overlay_dma_base = u64_to_flcn64(addr_code);
+ desc->argc = 1;
+ /* args are stored at the beginning of EMEM */
+ desc->argv = 0x01000000;
+}
+
+const struct acr_r352_ls_func
+acr_r361_ls_sec2_func = {
+ .load = acr_ls_ucode_load_sec2,
+ .generate_bl_desc = acr_r361_generate_sec2_bl_desc,
+ .bl_desc_size = sizeof(struct acr_r361_pmu_bl_desc),
+ .post_run = acr_ls_sec2_post_run,
+};
+
+
const struct acr_r352_func
acr_r361_func = {
+ .fixup_hs_desc = acr_r352_fixup_hs_desc,
.generate_hs_bl_desc = acr_r361_generate_hs_bl_desc,
.hs_bl_desc_size = sizeof(struct acr_r361_flcn_bl_desc),
.ls_ucode_img_load = acr_r352_ls_ucode_img_load,
.ls_func = {
[NVKM_SECBOOT_FALCON_FECS] = &acr_r361_ls_fecs_func,
[NVKM_SECBOOT_FALCON_GPCCS] = &acr_r361_ls_gpccs_func,
+ [NVKM_SECBOOT_FALCON_PMU] = &acr_r361_ls_pmu_func,
+ [NVKM_SECBOOT_FALCON_SEC2] = &acr_r361_ls_sec2_func,
},
};
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef __NVKM_SECBOOT_ACR_R361_H__
+#define __NVKM_SECBOOT_ACR_R361_H__
+
+#include "acr_r352.h"
+
+/**
+ * struct acr_r361_flcn_bl_desc - DMEM bootloader descriptor
+ * @signature: 16B signature for secure code. 0s if no secure code
+ * @ctx_dma: DMA context to be used by BL while loading code/data
+ * @code_dma_base: 256B-aligned Physical FB Address where code is located
+ * (falcon's $xcbase register)
+ * @non_sec_code_off: offset from code_dma_base where the non-secure code is
+ * located. The offset must be multiple of 256 to help perf
+ * @non_sec_code_size: the size of the nonSecure code part.
+ * @sec_code_off: offset from code_dma_base where the secure code is
+ * located. The offset must be multiple of 256 to help perf
+ * @sec_code_size: offset from code_dma_base where the secure code is
+ * located. The offset must be multiple of 256 to help perf
+ * @code_entry_point: code entry point which will be invoked by BL after
+ * code is loaded.
+ * @data_dma_base: 256B aligned Physical FB Address where data is located.
+ * (falcon's $xdbase register)
+ * @data_size: size of data block. Should be multiple of 256B
+ *
+ * Structure used by the bootloader to load the rest of the code. This has
+ * to be filled by host and copied into DMEM at offset provided in the
+ * hsflcn_bl_desc.bl_desc_dmem_load_off.
+ */
+struct acr_r361_flcn_bl_desc {
+ u32 reserved[4];
+ u32 signature[4];
+ u32 ctx_dma;
+ struct flcn_u64 code_dma_base;
+ u32 non_sec_code_off;
+ u32 non_sec_code_size;
+ u32 sec_code_off;
+ u32 sec_code_size;
+ u32 code_entry_point;
+ struct flcn_u64 data_dma_base;
+ u32 data_size;
+};
+
+void acr_r361_generate_hs_bl_desc(const struct hsf_load_header *, void *, u64);
+
+extern const struct acr_r352_ls_func acr_r361_ls_fecs_func;
+extern const struct acr_r352_ls_func acr_r361_ls_gpccs_func;
+extern const struct acr_r352_ls_func acr_r361_ls_pmu_func;
+extern const struct acr_r352_ls_func acr_r361_ls_sec2_func;
+
+#endif
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include "acr_r361.h"
+
+#include <core/gpuobj.h>
+
+/*
+ * r364 ACR: hsflcn_desc structure has changed to introduce the shadow_mem
+ * parameter.
+ */
+
+struct acr_r364_hsflcn_desc {
+ union {
+ u8 reserved_dmem[0x200];
+ u32 signatures[4];
+ } ucode_reserved_space;
+ u32 wpr_region_id;
+ u32 wpr_offset;
+ u32 mmu_memory_range;
+ struct {
+ u32 no_regions;
+ struct {
+ u32 start_addr;
+ u32 end_addr;
+ u32 region_id;
+ u32 read_mask;
+ u32 write_mask;
+ u32 client_mask;
+ u32 shadow_mem_start_addr;
+ } region_props[2];
+ } regions;
+ u32 ucode_blob_size;
+ u64 ucode_blob_base __aligned(8);
+ struct {
+ u32 vpr_enabled;
+ u32 vpr_start;
+ u32 vpr_end;
+ u32 hdcp_policies;
+ } vpr_desc;
+};
+
+static void
+acr_r364_fixup_hs_desc(struct acr_r352 *acr, struct nvkm_secboot *sb,
+ void *_desc)
+{
+ struct acr_r364_hsflcn_desc *desc = _desc;
+ struct nvkm_gpuobj *ls_blob = acr->ls_blob;
+
+ /* WPR region information if WPR is not fixed */
+ if (sb->wpr_size == 0) {
+ u64 wpr_start = ls_blob->addr;
+ u64 wpr_end = ls_blob->addr + ls_blob->size;
+
+ if (acr->func->shadow_blob)
+ wpr_start += ls_blob->size / 2;
+
+ desc->wpr_region_id = 1;
+ desc->regions.no_regions = 2;
+ desc->regions.region_props[0].start_addr = wpr_start >> 8;
+ desc->regions.region_props[0].end_addr = wpr_end >> 8;
+ desc->regions.region_props[0].region_id = 1;
+ desc->regions.region_props[0].read_mask = 0xf;
+ desc->regions.region_props[0].write_mask = 0xc;
+ desc->regions.region_props[0].client_mask = 0x2;
+ if (acr->func->shadow_blob)
+ desc->regions.region_props[0].shadow_mem_start_addr =
+ ls_blob->addr >> 8;
+ else
+ desc->regions.region_props[0].shadow_mem_start_addr = 0;
+ } else {
+ desc->ucode_blob_base = ls_blob->addr;
+ desc->ucode_blob_size = ls_blob->size;
+ }
+}
+
+const struct acr_r352_func
+acr_r364_func = {
+ .fixup_hs_desc = acr_r364_fixup_hs_desc,
+ .generate_hs_bl_desc = acr_r361_generate_hs_bl_desc,
+ .hs_bl_desc_size = sizeof(struct acr_r361_flcn_bl_desc),
+ .ls_ucode_img_load = acr_r352_ls_ucode_img_load,
+ .ls_fill_headers = acr_r352_ls_fill_headers,
+ .ls_write_wpr = acr_r352_ls_write_wpr,
+ .ls_func = {
+ [NVKM_SECBOOT_FALCON_FECS] = &acr_r361_ls_fecs_func,
+ [NVKM_SECBOOT_FALCON_GPCCS] = &acr_r361_ls_gpccs_func,
+ [NVKM_SECBOOT_FALCON_PMU] = &acr_r361_ls_pmu_func,
+ },
+};
+
+
+struct nvkm_acr *
+acr_r364_new(unsigned long managed_falcons)
+{
+ return acr_r352_new_(&acr_r364_func, NVKM_SECBOOT_FALCON_PMU,
+ managed_falcons);
+}
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include "acr_r367.h"
+#include "acr_r361.h"
+
+#include <core/gpuobj.h>
+
+/*
+ * r367 ACR: new LS signature format requires a rewrite of LS firmware and
+ * blob creation functions. Also the hsflcn_desc layout has changed slightly.
+ */
+
+#define LSF_LSB_DEPMAP_SIZE 11
+
+/**
+ * struct acr_r367_lsf_lsb_header - LS firmware header
+ *
+ * See also struct acr_r352_lsf_lsb_header for documentation.
+ */
+struct acr_r367_lsf_lsb_header {
+ /**
+ * LS falcon signatures
+ * @prd_keys: signature to use in production mode
+ * @dgb_keys: signature to use in debug mode
+ * @b_prd_present: whether the production key is present
+ * @b_dgb_present: whether the debug key is present
+ * @falcon_id: ID of the falcon the ucode applies to
+ */
+ struct {
+ u8 prd_keys[2][16];
+ u8 dbg_keys[2][16];
+ u32 b_prd_present;
+ u32 b_dbg_present;
+ u32 falcon_id;
+ u32 supports_versioning;
+ u32 version;
+ u32 depmap_count;
+ u8 depmap[LSF_LSB_DEPMAP_SIZE * 2 * 4];
+ u8 kdf[16];
+ } signature;
+ u32 ucode_off;
+ u32 ucode_size;
+ u32 data_size;
+ u32 bl_code_size;
+ u32 bl_imem_off;
+ u32 bl_data_off;
+ u32 bl_data_size;
+ u32 app_code_off;
+ u32 app_code_size;
+ u32 app_data_off;
+ u32 app_data_size;
+ u32 flags;
+};
+
+/**
+ * struct acr_r367_lsf_wpr_header - LS blob WPR Header
+ *
+ * See also struct acr_r352_lsf_wpr_header for documentation.
+ */
+struct acr_r367_lsf_wpr_header {
+ u32 falcon_id;
+ u32 lsb_offset;
+ u32 bootstrap_owner;
+ u32 lazy_bootstrap;
+ u32 bin_version;
+ u32 status;
+#define LSF_IMAGE_STATUS_NONE 0
+#define LSF_IMAGE_STATUS_COPY 1
+#define LSF_IMAGE_STATUS_VALIDATION_CODE_FAILED 2
+#define LSF_IMAGE_STATUS_VALIDATION_DATA_FAILED 3
+#define LSF_IMAGE_STATUS_VALIDATION_DONE 4
+#define LSF_IMAGE_STATUS_VALIDATION_SKIPPED 5
+#define LSF_IMAGE_STATUS_BOOTSTRAP_READY 6
+#define LSF_IMAGE_STATUS_REVOCATION_CHECK_FAILED 7
+};
+
+/**
+ * struct ls_ucode_img_r367 - ucode image augmented with r367 headers
+ */
+struct ls_ucode_img_r367 {
+ struct ls_ucode_img base;
+
+ struct acr_r367_lsf_wpr_header wpr_header;
+ struct acr_r367_lsf_lsb_header lsb_header;
+};
+#define ls_ucode_img_r367(i) container_of(i, struct ls_ucode_img_r367, base)
+
+struct ls_ucode_img *
+acr_r367_ls_ucode_img_load(const struct acr_r352 *acr,
+ enum nvkm_secboot_falcon falcon_id)
+{
+ const struct nvkm_subdev *subdev = acr->base.subdev;
+ struct ls_ucode_img_r367 *img;
+ int ret;
+
+ img = kzalloc(sizeof(*img), GFP_KERNEL);
+ if (!img)
+ return ERR_PTR(-ENOMEM);
+
+ img->base.falcon_id = falcon_id;
+
+ ret = acr->func->ls_func[falcon_id]->load(subdev, &img->base);
+ if (ret) {
+ kfree(img->base.ucode_data);
+ kfree(img->base.sig);
+ kfree(img);
+ return ERR_PTR(ret);
+ }
+
+ /* Check that the signature size matches our expectations... */
+ if (img->base.sig_size != sizeof(img->lsb_header.signature)) {
+ nvkm_error(subdev, "invalid signature size for %s falcon!\n",
+ nvkm_secboot_falcon_name[falcon_id]);
+ return ERR_PTR(-EINVAL);
+ }
+
+ /* Copy signature to the right place */
+ memcpy(&img->lsb_header.signature, img->base.sig, img->base.sig_size);
+
+ /* not needed? the signature should already have the right value */
+ img->lsb_header.signature.falcon_id = falcon_id;
+
+ return &img->base;
+}
+
+#define LSF_LSB_HEADER_ALIGN 256
+#define LSF_BL_DATA_ALIGN 256
+#define LSF_BL_DATA_SIZE_ALIGN 256
+#define LSF_BL_CODE_SIZE_ALIGN 256
+#define LSF_UCODE_DATA_ALIGN 4096
+
+static u32
+acr_r367_ls_img_fill_headers(struct acr_r352 *acr,
+ struct ls_ucode_img_r367 *img, u32 offset)
+{
+ struct ls_ucode_img *_img = &img->base;
+ struct acr_r367_lsf_wpr_header *whdr = &img->wpr_header;
+ struct acr_r367_lsf_lsb_header *lhdr = &img->lsb_header;
+ struct ls_ucode_img_desc *desc = &_img->ucode_desc;
+ const struct acr_r352_ls_func *func =
+ acr->func->ls_func[_img->falcon_id];
+
+ /* Fill WPR header */
+ whdr->falcon_id = _img->falcon_id;
+ whdr->bootstrap_owner = acr->base.boot_falcon;
+ whdr->bin_version = lhdr->signature.version;
+ whdr->status = LSF_IMAGE_STATUS_COPY;
+
+ /* Skip bootstrapping falcons started by someone else than ACR */
+ if (acr->lazy_bootstrap & BIT(_img->falcon_id))
+ whdr->lazy_bootstrap = 1;
+
+ /* Align, save off, and include an LSB header size */
+ offset = ALIGN(offset, LSF_LSB_HEADER_ALIGN);
+ whdr->lsb_offset = offset;
+ offset += sizeof(*lhdr);
+
+ /*
+ * Align, save off, and include the original (static) ucode
+ * image size
+ */
+ offset = ALIGN(offset, LSF_UCODE_DATA_ALIGN);
+ _img->ucode_off = lhdr->ucode_off = offset;
+ offset += _img->ucode_size;
+
+ /*
+ * For falcons that use a boot loader (BL), we append a loader
+ * desc structure on the end of the ucode image and consider
+ * this the boot loader data. The host will then copy the loader
+ * desc args to this space within the WPR region (before locking
+ * down) and the HS bin will then copy them to DMEM 0 for the
+ * loader.
+ */
+ lhdr->bl_code_size = ALIGN(desc->bootloader_size,
+ LSF_BL_CODE_SIZE_ALIGN);
+ lhdr->ucode_size = ALIGN(desc->app_resident_data_offset,
+ LSF_BL_CODE_SIZE_ALIGN) + lhdr->bl_code_size;
+ lhdr->data_size = ALIGN(desc->app_size, LSF_BL_CODE_SIZE_ALIGN) +
+ lhdr->bl_code_size - lhdr->ucode_size;
+ /*
+ * Though the BL is located at 0th offset of the image, the VA
+ * is different to make sure that it doesn't collide the actual
+ * OS VA range
+ */
+ lhdr->bl_imem_off = desc->bootloader_imem_offset;
+ lhdr->app_code_off = desc->app_start_offset +
+ desc->app_resident_code_offset;
+ lhdr->app_code_size = desc->app_resident_code_size;
+ lhdr->app_data_off = desc->app_start_offset +
+ desc->app_resident_data_offset;
+ lhdr->app_data_size = desc->app_resident_data_size;
+
+ lhdr->flags = func->lhdr_flags;
+ if (_img->falcon_id == acr->base.boot_falcon)
+ lhdr->flags |= LSF_FLAG_DMACTL_REQ_CTX;
+
+ /* Align and save off BL descriptor size */
+ lhdr->bl_data_size = ALIGN(func->bl_desc_size, LSF_BL_DATA_SIZE_ALIGN);
+
+ /*
+ * Align, save off, and include the additional BL data
+ */
+ offset = ALIGN(offset, LSF_BL_DATA_ALIGN);
+ lhdr->bl_data_off = offset;
+ offset += lhdr->bl_data_size;
+
+ return offset;
+}
+
+int
+acr_r367_ls_fill_headers(struct acr_r352 *acr, struct list_head *imgs)
+{
+ struct ls_ucode_img_r367 *img;
+ struct list_head *l;
+ u32 count = 0;
+ u32 offset;
+
+ /* Count the number of images to manage */
+ list_for_each(l, imgs)
+ count++;
+
+ /*
+ * Start with an array of WPR headers at the base of the WPR.
+ * The expectation here is that the secure falcon will do a single DMA
+ * read of this array and cache it internally so it's ok to pack these.
+ * Also, we add 1 to the falcon count to indicate the end of the array.
+ */
+ offset = sizeof(img->wpr_header) * (count + 1);
+
+ /*
+ * Walk the managed falcons, accounting for the LSB structs
+ * as well as the ucode images.
+ */
+ list_for_each_entry(img, imgs, base.node) {
+ offset = acr_r367_ls_img_fill_headers(acr, img, offset);
+ }
+
+ return offset;
+}
+
+int
+acr_r367_ls_write_wpr(struct acr_r352 *acr, struct list_head *imgs,
+ struct nvkm_gpuobj *wpr_blob, u64 wpr_addr)
+{
+ struct ls_ucode_img *_img;
+ u32 pos = 0;
+
+ nvkm_kmap(wpr_blob);
+
+ list_for_each_entry(_img, imgs, node) {
+ struct ls_ucode_img_r367 *img = ls_ucode_img_r367(_img);
+ const struct acr_r352_ls_func *ls_func =
+ acr->func->ls_func[_img->falcon_id];
+ u8 gdesc[ls_func->bl_desc_size];
+
+ nvkm_gpuobj_memcpy_to(wpr_blob, pos, &img->wpr_header,
+ sizeof(img->wpr_header));
+
+ nvkm_gpuobj_memcpy_to(wpr_blob, img->wpr_header.lsb_offset,
+ &img->lsb_header, sizeof(img->lsb_header));
+
+ /* Generate and write BL descriptor */
+ memset(gdesc, 0, ls_func->bl_desc_size);
+ ls_func->generate_bl_desc(&acr->base, _img, wpr_addr, gdesc);
+
+ nvkm_gpuobj_memcpy_to(wpr_blob, img->lsb_header.bl_data_off,
+ gdesc, ls_func->bl_desc_size);
+
+ /* Copy ucode */
+ nvkm_gpuobj_memcpy_to(wpr_blob, img->lsb_header.ucode_off,
+ _img->ucode_data, _img->ucode_size);
+
+ pos += sizeof(img->wpr_header);
+ }
+
+ nvkm_wo32(wpr_blob, pos, NVKM_SECBOOT_FALCON_INVALID);
+
+ nvkm_done(wpr_blob);
+
+ return 0;
+}
+
+struct acr_r367_hsflcn_desc {
+ u8 reserved_dmem[0x200];
+ u32 signatures[4];
+ u32 wpr_region_id;
+ u32 wpr_offset;
+ u32 mmu_memory_range;
+#define FLCN_ACR_MAX_REGIONS 2
+ struct {
+ u32 no_regions;
+ struct {
+ u32 start_addr;
+ u32 end_addr;
+ u32 region_id;
+ u32 read_mask;
+ u32 write_mask;
+ u32 client_mask;
+ u32 shadow_mem_start_addr;
+ } region_props[FLCN_ACR_MAX_REGIONS];
+ } regions;
+ u32 ucode_blob_size;
+ u64 ucode_blob_base __aligned(8);
+ struct {
+ u32 vpr_enabled;
+ u32 vpr_start;
+ u32 vpr_end;
+ u32 hdcp_policies;
+ } vpr_desc;
+};
+
+void
+acr_r367_fixup_hs_desc(struct acr_r352 *acr, struct nvkm_secboot *sb,
+ void *_desc)
+{
+ struct acr_r367_hsflcn_desc *desc = _desc;
+ struct nvkm_gpuobj *ls_blob = acr->ls_blob;
+
+ /* WPR region information if WPR is not fixed */
+ if (sb->wpr_size == 0) {
+ u64 wpr_start = ls_blob->addr;
+ u64 wpr_end = ls_blob->addr + ls_blob->size;
+
+ if (acr->func->shadow_blob)
+ wpr_start += ls_blob->size / 2;
+
+ desc->wpr_region_id = 1;
+ desc->regions.no_regions = 2;
+ desc->regions.region_props[0].start_addr = wpr_start >> 8;
+ desc->regions.region_props[0].end_addr = wpr_end >> 8;
+ desc->regions.region_props[0].region_id = 1;
+ desc->regions.region_props[0].read_mask = 0xf;
+ desc->regions.region_props[0].write_mask = 0xc;
+ desc->regions.region_props[0].client_mask = 0x2;
+ if (acr->func->shadow_blob)
+ desc->regions.region_props[0].shadow_mem_start_addr =
+ ls_blob->addr >> 8;
+ else
+ desc->regions.region_props[0].shadow_mem_start_addr = 0;
+ } else {
+ desc->ucode_blob_base = ls_blob->addr;
+ desc->ucode_blob_size = ls_blob->size;
+ }
+}
+
+const struct acr_r352_func
+acr_r367_func = {
+ .fixup_hs_desc = acr_r367_fixup_hs_desc,
+ .generate_hs_bl_desc = acr_r361_generate_hs_bl_desc,
+ .hs_bl_desc_size = sizeof(struct acr_r361_flcn_bl_desc),
+ .shadow_blob = true,
+ .ls_ucode_img_load = acr_r367_ls_ucode_img_load,
+ .ls_fill_headers = acr_r367_ls_fill_headers,
+ .ls_write_wpr = acr_r367_ls_write_wpr,
+ .ls_func = {
+ [NVKM_SECBOOT_FALCON_FECS] = &acr_r361_ls_fecs_func,
+ [NVKM_SECBOOT_FALCON_GPCCS] = &acr_r361_ls_gpccs_func,
+ [NVKM_SECBOOT_FALCON_PMU] = &acr_r361_ls_pmu_func,
+ [NVKM_SECBOOT_FALCON_SEC2] = &acr_r361_ls_sec2_func,
+ },
+};
+
+struct nvkm_acr *
+acr_r367_new(enum nvkm_secboot_falcon boot_falcon,
+ unsigned long managed_falcons)
+{
+ return acr_r352_new_(&acr_r367_func, boot_falcon, managed_falcons);
+}
--- /dev/null
+/*
+ * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef __NVKM_SECBOOT_ACR_R367_H__
+#define __NVKM_SECBOOT_ACR_R367_H__
+
+#include "acr_r352.h"
+
+void acr_r367_fixup_hs_desc(struct acr_r352 *, struct nvkm_secboot *, void *);
+
+struct ls_ucode_img *acr_r367_ls_ucode_img_load(const struct acr_r352 *,
+ enum nvkm_secboot_falcon);
+int acr_r367_ls_fill_headers(struct acr_r352 *, struct list_head *);
+int acr_r367_ls_write_wpr(struct acr_r352 *, struct list_head *,
+ struct nvkm_gpuobj *, u64);
+#endif
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include "acr_r367.h"
+
+#include <engine/falcon.h>
+#include <core/msgqueue.h>
+#include <subdev/pmu.h>
+
+/*
+ * r375 ACR: similar to r367, but with a unified bootloader descriptor
+ * structure for GR and PMU falcons.
+ */
+
+/* Same as acr_r361_flcn_bl_desc, plus argc/argv */
+struct acr_r375_flcn_bl_desc {
+ u32 reserved[4];
+ u32 signature[4];
+ u32 ctx_dma;
+ struct flcn_u64 code_dma_base;
+ u32 non_sec_code_off;
+ u32 non_sec_code_size;
+ u32 sec_code_off;
+ u32 sec_code_size;
+ u32 code_entry_point;
+ struct flcn_u64 data_dma_base;
+ u32 data_size;
+ u32 argc;
+ u32 argv;
+};
+
+static void
+acr_r375_generate_flcn_bl_desc(const struct nvkm_acr *acr,
+ const struct ls_ucode_img *img, u64 wpr_addr,
+ void *_desc)
+{
+ struct acr_r375_flcn_bl_desc *desc = _desc;
+ const struct ls_ucode_img_desc *pdesc = &img->ucode_desc;
+ u64 base, addr_code, addr_data;
+
+ base = wpr_addr + img->ucode_off + pdesc->app_start_offset;
+ addr_code = base + pdesc->app_resident_code_offset;
+ addr_data = base + pdesc->app_resident_data_offset;
+
+ desc->ctx_dma = FALCON_DMAIDX_UCODE;
+ desc->code_dma_base = u64_to_flcn64(addr_code);
+ desc->non_sec_code_off = pdesc->app_resident_code_offset;
+ desc->non_sec_code_size = pdesc->app_resident_code_size;
+ desc->code_entry_point = pdesc->app_imem_entry;
+ desc->data_dma_base = u64_to_flcn64(addr_data);
+ desc->data_size = pdesc->app_resident_data_size;
+}
+
+static void
+acr_r375_generate_hs_bl_desc(const struct hsf_load_header *hdr, void *_bl_desc,
+ u64 offset)
+{
+ struct acr_r375_flcn_bl_desc *bl_desc = _bl_desc;
+
+ bl_desc->ctx_dma = FALCON_DMAIDX_VIRT;
+ bl_desc->non_sec_code_off = hdr->non_sec_code_off;
+ bl_desc->non_sec_code_size = hdr->non_sec_code_size;
+ bl_desc->sec_code_off = hsf_load_header_app_off(hdr, 0);
+ bl_desc->sec_code_size = hsf_load_header_app_size(hdr, 0);
+ bl_desc->code_entry_point = 0;
+ bl_desc->code_dma_base = u64_to_flcn64(offset);
+ bl_desc->data_dma_base = u64_to_flcn64(offset + hdr->data_dma_base);
+ bl_desc->data_size = hdr->data_size;
+}
+
+const struct acr_r352_ls_func
+acr_r375_ls_fecs_func = {
+ .load = acr_ls_ucode_load_fecs,
+ .generate_bl_desc = acr_r375_generate_flcn_bl_desc,
+ .bl_desc_size = sizeof(struct acr_r375_flcn_bl_desc),
+};
+
+const struct acr_r352_ls_func
+acr_r375_ls_gpccs_func = {
+ .load = acr_ls_ucode_load_gpccs,
+ .generate_bl_desc = acr_r375_generate_flcn_bl_desc,
+ .bl_desc_size = sizeof(struct acr_r375_flcn_bl_desc),
+ /* GPCCS will be loaded using PRI */
+ .lhdr_flags = LSF_FLAG_FORCE_PRIV_LOAD,
+};
+
+
+static void
+acr_r375_generate_pmu_bl_desc(const struct nvkm_acr *acr,
+ const struct ls_ucode_img *img, u64 wpr_addr,
+ void *_desc)
+{
+ const struct ls_ucode_img_desc *pdesc = &img->ucode_desc;
+ const struct nvkm_pmu *pmu = acr->subdev->device->pmu;
+ struct acr_r375_flcn_bl_desc *desc = _desc;
+ u64 base, addr_code, addr_data;
+ u32 addr_args;
+
+ base = wpr_addr + img->ucode_off + pdesc->app_start_offset;
+ addr_code = base + pdesc->app_resident_code_offset;
+ addr_data = base + pdesc->app_resident_data_offset;
+ addr_args = pmu->falcon->data.limit;
+ addr_args -= NVKM_MSGQUEUE_CMDLINE_SIZE;
+
+ desc->ctx_dma = FALCON_DMAIDX_UCODE;
+ desc->code_dma_base = u64_to_flcn64(addr_code);
+ desc->non_sec_code_off = pdesc->app_resident_code_offset;
+ desc->non_sec_code_size = pdesc->app_resident_code_size;
+ desc->code_entry_point = pdesc->app_imem_entry;
+ desc->data_dma_base = u64_to_flcn64(addr_data);
+ desc->data_size = pdesc->app_resident_data_size;
+ desc->argc = 1;
+ desc->argv = addr_args;
+}
+
+const struct acr_r352_ls_func
+acr_r375_ls_pmu_func = {
+ .load = acr_ls_ucode_load_pmu,
+ .generate_bl_desc = acr_r375_generate_pmu_bl_desc,
+ .bl_desc_size = sizeof(struct acr_r375_flcn_bl_desc),
+ .post_run = acr_ls_pmu_post_run,
+};
+
+
+const struct acr_r352_func
+acr_r375_func = {
+ .fixup_hs_desc = acr_r367_fixup_hs_desc,
+ .generate_hs_bl_desc = acr_r375_generate_hs_bl_desc,
+ .hs_bl_desc_size = sizeof(struct acr_r375_flcn_bl_desc),
+ .shadow_blob = true,
+ .ls_ucode_img_load = acr_r367_ls_ucode_img_load,
+ .ls_fill_headers = acr_r367_ls_fill_headers,
+ .ls_write_wpr = acr_r367_ls_write_wpr,
+ .ls_func = {
+ [NVKM_SECBOOT_FALCON_FECS] = &acr_r375_ls_fecs_func,
+ [NVKM_SECBOOT_FALCON_GPCCS] = &acr_r375_ls_gpccs_func,
+ [NVKM_SECBOOT_FALCON_PMU] = &acr_r375_ls_pmu_func,
+ },
+};
+
+struct nvkm_acr *
+acr_r375_new(enum nvkm_secboot_falcon boot_falcon,
+ unsigned long managed_falcons)
+{
+ return acr_r352_new_(&acr_r375_func, boot_falcon, managed_falcons);
+}
#include <subdev/mc.h>
#include <subdev/timer.h>
#include <subdev/pmu.h>
+#include <engine/sec2.h>
const char *
nvkm_secboot_falcon_name[] = {
[NVKM_SECBOOT_FALCON_RESERVED] = "<reserved>",
[NVKM_SECBOOT_FALCON_FECS] = "FECS",
[NVKM_SECBOOT_FALCON_GPCCS] = "GPCCS",
+ [NVKM_SECBOOT_FALCON_SEC2] = "SEC2",
[NVKM_SECBOOT_FALCON_END] = "<invalid>",
};
/**
switch (sb->acr->boot_falcon) {
case NVKM_SECBOOT_FALCON_PMU:
- sb->boot_falcon = subdev->device->pmu->falcon;
+ sb->halt_falcon = sb->boot_falcon = subdev->device->pmu->falcon;
+ break;
+ case NVKM_SECBOOT_FALCON_SEC2:
+ /* we must keep SEC2 alive forever since ACR will run on it */
+ nvkm_engine_ref(&subdev->device->sec2->engine);
+ sb->boot_falcon = subdev->device->sec2->falcon;
+ sb->halt_falcon = subdev->device->pmu->falcon;
break;
default:
nvkm_error(subdev, "Unmanaged boot falcon %s!\n",
nvkm_secboot_falcon_name[sb->acr->boot_falcon]);
return -EINVAL;
}
+ nvkm_debug(subdev, "using %s falcon for ACR\n", sb->boot_falcon->name);
/* Call chip-specific init function */
if (sb->func->oneinit)
*
*/
int
-gm200_secboot_run_blob(struct nvkm_secboot *sb, struct nvkm_gpuobj *blob)
+gm200_secboot_run_blob(struct nvkm_secboot *sb, struct nvkm_gpuobj *blob,
+ struct nvkm_falcon *falcon)
{
struct gm200_secboot *gsb = gm200_secboot(sb);
struct nvkm_subdev *subdev = &gsb->base.subdev;
- struct nvkm_falcon *falcon = gsb->base.boot_falcon;
struct nvkm_vma vma;
+ u32 start_address;
int ret;
ret = nvkm_falcon_get(falcon, subdev);
nvkm_falcon_bind_context(falcon, gsb->inst);
/* Load the HS bootloader into the falcon's IMEM/DMEM */
- ret = sb->acr->func->load(sb->acr, &gsb->base, blob, vma.offset);
- if (ret)
+ ret = sb->acr->func->load(sb->acr, falcon, blob, vma.offset);
+ if (ret < 0)
goto end;
+ start_address = ret;
+
/* Disable interrupts as we will poll for the HALT bit */
nvkm_mc_intr_mask(sb->subdev.device, falcon->owner->index, false);
nvkm_falcon_wr32(falcon, 0x040, 0xdeada5a5);
/* Start the HS bootloader */
- nvkm_falcon_set_start_addr(falcon, sb->acr->start_address);
+ nvkm_falcon_set_start_addr(falcon, start_address);
nvkm_falcon_start(falcon);
ret = nvkm_falcon_wait_for_halt(falcon, 100);
if (ret)
goto end;
- /* If mailbox register contains an error code, then ACR has failed */
+ /*
+ * The mailbox register contains the (positive) error code - return this
+ * to the caller
+ */
ret = nvkm_falcon_rd32(falcon, 0x040);
- if (ret) {
- nvkm_error(subdev, "ACR boot failed, ret 0x%08x", ret);
- ret = -EINVAL;
- goto end;
- }
end:
/* Reenable interrupts */
int gm200_secboot_oneinit(struct nvkm_secboot *);
int gm200_secboot_fini(struct nvkm_secboot *, bool);
void *gm200_secboot_dtor(struct nvkm_secboot *);
-int gm200_secboot_run_blob(struct nvkm_secboot *, struct nvkm_gpuobj *);
+int gm200_secboot_run_blob(struct nvkm_secboot *, struct nvkm_gpuobj *,
+ struct nvkm_falcon *);
#endif
struct gm200_secboot *gsb;
struct nvkm_acr *acr;
- acr = acr_r352_new(BIT(NVKM_SECBOOT_FALCON_FECS));
+ acr = acr_r352_new(BIT(NVKM_SECBOOT_FALCON_FECS) |
+ BIT(NVKM_SECBOOT_FALCON_PMU));
if (IS_ERR(acr))
return PTR_ERR(acr);
+ /* Support the initial GM20B firmware release without PMU */
+ acr->optional_falcons = BIT(NVKM_SECBOOT_FALCON_PMU);
gsb = kzalloc(sizeof(*gsb), GFP_KERNEL);
if (!gsb) {
MODULE_FIRMWARE("nvidia/gm20b/gr/sw_nonctx.bin");
MODULE_FIRMWARE("nvidia/gm20b/gr/sw_bundle_init.bin");
MODULE_FIRMWARE("nvidia/gm20b/gr/sw_method_init.bin");
+MODULE_FIRMWARE("nvidia/gm20b/pmu/desc.bin");
+MODULE_FIRMWARE("nvidia/gm20b/pmu/image.bin");
+MODULE_FIRMWARE("nvidia/gm20b/pmu/sig.bin");
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include "acr.h"
+#include "gm200.h"
+
+#include "ls_ucode.h"
+#include "hs_ucode.h"
+#include <subdev/mc.h>
+#include <subdev/timer.h>
+#include <engine/falcon.h>
+#include <engine/nvdec.h>
+
+static bool
+gp102_secboot_scrub_required(struct nvkm_secboot *sb)
+{
+ struct nvkm_subdev *subdev = &sb->subdev;
+ struct nvkm_device *device = subdev->device;
+ u32 reg;
+
+ nvkm_wr32(device, 0x100cd0, 0x2);
+ reg = nvkm_rd32(device, 0x100cd0);
+
+ return (reg & BIT(4));
+}
+
+static int
+gp102_run_secure_scrub(struct nvkm_secboot *sb)
+{
+ struct nvkm_subdev *subdev = &sb->subdev;
+ struct nvkm_device *device = subdev->device;
+ struct nvkm_engine *engine;
+ struct nvkm_falcon *falcon;
+ void *scrub_image;
+ struct fw_bin_header *hsbin_hdr;
+ struct hsf_fw_header *fw_hdr;
+ struct hsf_load_header *lhdr;
+ void *scrub_data;
+ int ret;
+
+ nvkm_debug(subdev, "running VPR scrubber binary on NVDEC...\n");
+
+ engine = nvkm_engine_ref(&device->nvdec->engine);
+ if (IS_ERR(engine))
+ return PTR_ERR(engine);
+ falcon = device->nvdec->falcon;
+
+ nvkm_falcon_get(falcon, &sb->subdev);
+
+ scrub_image = hs_ucode_load_blob(subdev, falcon, "nvdec/scrubber");
+ if (IS_ERR(scrub_image))
+ return PTR_ERR(scrub_image);
+
+ nvkm_falcon_reset(falcon);
+ nvkm_falcon_bind_context(falcon, NULL);
+
+ hsbin_hdr = scrub_image;
+ fw_hdr = scrub_image + hsbin_hdr->header_offset;
+ lhdr = scrub_image + fw_hdr->hdr_offset;
+ scrub_data = scrub_image + hsbin_hdr->data_offset;
+
+ nvkm_falcon_load_imem(falcon, scrub_data, lhdr->non_sec_code_off,
+ lhdr->non_sec_code_size,
+ lhdr->non_sec_code_off >> 8, 0, false);
+ nvkm_falcon_load_imem(falcon, scrub_data + lhdr->apps[0],
+ ALIGN(lhdr->apps[0], 0x100),
+ lhdr->apps[1],
+ lhdr->apps[0] >> 8, 0, true);
+ nvkm_falcon_load_dmem(falcon, scrub_data + lhdr->data_dma_base, 0,
+ lhdr->data_size, 0);
+
+ kfree(scrub_image);
+
+ nvkm_falcon_set_start_addr(falcon, 0x0);
+ nvkm_falcon_start(falcon);
+
+ ret = nvkm_falcon_wait_for_halt(falcon, 500);
+ if (ret < 0) {
+ nvkm_error(subdev, "failed to run VPR scrubber binary!\n");
+ ret = -ETIMEDOUT;
+ goto end;
+ }
+
+ /* put nvdec in clean state - without reset it will remain in HS mode */
+ nvkm_falcon_reset(falcon);
+
+ if (gp102_secboot_scrub_required(sb)) {
+ nvkm_error(subdev, "VPR scrubber binary failed!\n");
+ ret = -EINVAL;
+ goto end;
+ }
+
+ nvkm_debug(subdev, "VPR scrub successfully completed\n");
+
+end:
+ nvkm_falcon_put(falcon, &sb->subdev);
+ nvkm_engine_unref(&engine);
+ return ret;
+}
+
+static int
+gp102_secboot_run_blob(struct nvkm_secboot *sb, struct nvkm_gpuobj *blob,
+ struct nvkm_falcon *falcon)
+{
+ int ret;
+
+ /* make sure the VPR region is unlocked */
+ if (gp102_secboot_scrub_required(sb)) {
+ ret = gp102_run_secure_scrub(sb);
+ if (ret)
+ return ret;
+ }
+
+ return gm200_secboot_run_blob(sb, blob, falcon);
+}
+
+static const struct nvkm_secboot_func
+gp102_secboot = {
+ .dtor = gm200_secboot_dtor,
+ .oneinit = gm200_secboot_oneinit,
+ .fini = gm200_secboot_fini,
+ .run_blob = gp102_secboot_run_blob,
+};
+
+int
+gp102_secboot_new(struct nvkm_device *device, int index,
+ struct nvkm_secboot **psb)
+{
+ int ret;
+ struct gm200_secboot *gsb;
+ struct nvkm_acr *acr;
+
+ acr = acr_r367_new(NVKM_SECBOOT_FALCON_SEC2,
+ BIT(NVKM_SECBOOT_FALCON_FECS) |
+ BIT(NVKM_SECBOOT_FALCON_GPCCS) |
+ BIT(NVKM_SECBOOT_FALCON_SEC2));
+ if (IS_ERR(acr))
+ return PTR_ERR(acr);
+
+ gsb = kzalloc(sizeof(*gsb), GFP_KERNEL);
+ if (!gsb) {
+ psb = NULL;
+ return -ENOMEM;
+ }
+ *psb = &gsb->base;
+
+ ret = nvkm_secboot_ctor(&gp102_secboot, acr, device, index, &gsb->base);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+MODULE_FIRMWARE("nvidia/gp102/acr/bl.bin");
+MODULE_FIRMWARE("nvidia/gp102/acr/unload_bl.bin");
+MODULE_FIRMWARE("nvidia/gp102/acr/ucode_load.bin");
+MODULE_FIRMWARE("nvidia/gp102/acr/ucode_unload.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/fecs_bl.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/fecs_inst.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/fecs_data.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/fecs_sig.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/gpccs_bl.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/gpccs_inst.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/gpccs_data.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/gpccs_sig.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/sw_ctx.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/sw_nonctx.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/sw_bundle_init.bin");
+MODULE_FIRMWARE("nvidia/gp102/gr/sw_method_init.bin");
+MODULE_FIRMWARE("nvidia/gp102/nvdec/scrubber.bin");
+MODULE_FIRMWARE("nvidia/gp102/sec2/desc.bin");
+MODULE_FIRMWARE("nvidia/gp102/sec2/image.bin");
+MODULE_FIRMWARE("nvidia/gp102/sec2/sig.bin");
+MODULE_FIRMWARE("nvidia/gp104/acr/bl.bin");
+MODULE_FIRMWARE("nvidia/gp104/acr/unload_bl.bin");
+MODULE_FIRMWARE("nvidia/gp104/acr/ucode_load.bin");
+MODULE_FIRMWARE("nvidia/gp104/acr/ucode_unload.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/fecs_bl.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/fecs_inst.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/fecs_data.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/fecs_sig.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/gpccs_bl.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/gpccs_inst.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/gpccs_data.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/gpccs_sig.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/sw_ctx.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/sw_nonctx.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/sw_bundle_init.bin");
+MODULE_FIRMWARE("nvidia/gp104/gr/sw_method_init.bin");
+MODULE_FIRMWARE("nvidia/gp104/nvdec/scrubber.bin");
+MODULE_FIRMWARE("nvidia/gp104/sec2/desc.bin");
+MODULE_FIRMWARE("nvidia/gp104/sec2/image.bin");
+MODULE_FIRMWARE("nvidia/gp104/sec2/sig.bin");
+MODULE_FIRMWARE("nvidia/gp106/acr/bl.bin");
+MODULE_FIRMWARE("nvidia/gp106/acr/unload_bl.bin");
+MODULE_FIRMWARE("nvidia/gp106/acr/ucode_load.bin");
+MODULE_FIRMWARE("nvidia/gp106/acr/ucode_unload.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/fecs_bl.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/fecs_inst.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/fecs_data.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/fecs_sig.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/gpccs_bl.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/gpccs_inst.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/gpccs_data.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/gpccs_sig.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/sw_ctx.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/sw_nonctx.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/sw_bundle_init.bin");
+MODULE_FIRMWARE("nvidia/gp106/gr/sw_method_init.bin");
+MODULE_FIRMWARE("nvidia/gp106/nvdec/scrubber.bin");
+MODULE_FIRMWARE("nvidia/gp106/sec2/desc.bin");
+MODULE_FIRMWARE("nvidia/gp106/sec2/image.bin");
+MODULE_FIRMWARE("nvidia/gp106/sec2/sig.bin");
+MODULE_FIRMWARE("nvidia/gp107/acr/bl.bin");
+MODULE_FIRMWARE("nvidia/gp107/acr/unload_bl.bin");
+MODULE_FIRMWARE("nvidia/gp107/acr/ucode_load.bin");
+MODULE_FIRMWARE("nvidia/gp107/acr/ucode_unload.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/fecs_bl.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/fecs_inst.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/fecs_data.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/fecs_sig.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/gpccs_bl.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/gpccs_inst.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/gpccs_data.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/gpccs_sig.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/sw_ctx.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/sw_nonctx.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/sw_bundle_init.bin");
+MODULE_FIRMWARE("nvidia/gp107/gr/sw_method_init.bin");
+MODULE_FIRMWARE("nvidia/gp107/nvdec/scrubber.bin");
+MODULE_FIRMWARE("nvidia/gp107/sec2/desc.bin");
+MODULE_FIRMWARE("nvidia/gp107/sec2/image.bin");
+MODULE_FIRMWARE("nvidia/gp107/sec2/sig.bin");
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include "hs_ucode.h"
+#include "ls_ucode.h"
+#include "acr.h"
+
+#include <engine/falcon.h>
+
+/**
+ * hs_ucode_patch_signature() - patch HS blob with correct signature for
+ * specified falcon.
+ */
+static void
+hs_ucode_patch_signature(const struct nvkm_falcon *falcon, void *acr_image,
+ bool new_format)
+{
+ struct fw_bin_header *hsbin_hdr = acr_image;
+ struct hsf_fw_header *fw_hdr = acr_image + hsbin_hdr->header_offset;
+ void *hs_data = acr_image + hsbin_hdr->data_offset;
+ void *sig;
+ u32 sig_size;
+ u32 patch_loc, patch_sig;
+
+ /*
+ * I had the brilliant idea to "improve" the binary format by
+ * removing this useless indirection. However to make NVIDIA files
+ * directly compatible, let's support both format.
+ */
+ if (new_format) {
+ patch_loc = fw_hdr->patch_loc;
+ patch_sig = fw_hdr->patch_sig;
+ } else {
+ patch_loc = *(u32 *)(acr_image + fw_hdr->patch_loc);
+ patch_sig = *(u32 *)(acr_image + fw_hdr->patch_sig);
+ }
+
+ /* Falcon in debug or production mode? */
+ if (falcon->debug) {
+ sig = acr_image + fw_hdr->sig_dbg_offset;
+ sig_size = fw_hdr->sig_dbg_size;
+ } else {
+ sig = acr_image + fw_hdr->sig_prod_offset;
+ sig_size = fw_hdr->sig_prod_size;
+ }
+
+ /* Patch signature */
+ memcpy(hs_data + patch_loc, sig + patch_sig, sig_size);
+}
+
+void *
+hs_ucode_load_blob(struct nvkm_subdev *subdev, const struct nvkm_falcon *falcon,
+ const char *fw)
+{
+ void *acr_image;
+ bool new_format;
+
+ acr_image = nvkm_acr_load_firmware(subdev, fw, 0);
+ if (IS_ERR(acr_image))
+ return acr_image;
+
+ /* detect the format to define how signature should be patched */
+ switch (((u32 *)acr_image)[0]) {
+ case 0x3b1d14f0:
+ new_format = true;
+ break;
+ case 0x000010de:
+ new_format = false;
+ break;
+ default:
+ nvkm_error(subdev, "unknown header for HS blob %s\n", fw);
+ return ERR_PTR(-EINVAL);
+ }
+
+ hs_ucode_patch_signature(falcon, acr_image, new_format);
+
+ return acr_image;
+}
--- /dev/null
+/*
+ * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef __NVKM_SECBOOT_HS_UCODE_H__
+#define __NVKM_SECBOOT_HS_UCODE_H__
+
+#include <core/os.h>
+#include <core/subdev.h>
+
+struct nvkm_falcon;
+
+/**
+ * struct hsf_fw_header - HS firmware descriptor
+ * @sig_dbg_offset: offset of the debug signature
+ * @sig_dbg_size: size of the debug signature
+ * @sig_prod_offset: offset of the production signature
+ * @sig_prod_size: size of the production signature
+ * @patch_loc: offset of the offset (sic) of where the signature is
+ * @patch_sig: offset of the offset (sic) to add to sig_*_offset
+ * @hdr_offset: offset of the load header (see struct hs_load_header)
+ * @hdr_size: size of above header
+ *
+ * This structure is embedded in the HS firmware image at
+ * hs_bin_hdr.header_offset.
+ */
+struct hsf_fw_header {
+ u32 sig_dbg_offset;
+ u32 sig_dbg_size;
+ u32 sig_prod_offset;
+ u32 sig_prod_size;
+ u32 patch_loc;
+ u32 patch_sig;
+ u32 hdr_offset;
+ u32 hdr_size;
+};
+
+/**
+ * struct hsf_load_header - HS firmware load header
+ */
+struct hsf_load_header {
+ u32 non_sec_code_off;
+ u32 non_sec_code_size;
+ u32 data_dma_base;
+ u32 data_size;
+ u32 num_apps;
+ /*
+ * Organized as follows:
+ * - app0_code_off
+ * - app1_code_off
+ * - ...
+ * - appn_code_off
+ * - app0_code_size
+ * - app1_code_size
+ * - ...
+ */
+ u32 apps[0];
+};
+
+void *hs_ucode_load_blob(struct nvkm_subdev *, const struct nvkm_falcon *,
+ const char *);
+
+#endif
#include <core/subdev.h>
#include <subdev/secboot.h>
+struct nvkm_acr;
/**
* struct ls_ucode_img_desc - descriptor of firmware image
* @ucode_desc: loaded or generated map of ucode_data
* @ucode_data: firmware payload (code and data)
* @ucode_size: size in bytes of data in ucode_data
+ * @ucode_off: offset of the ucode in ucode_data
* @sig: signature for this firmware
* @sig:size: size of the signature in bytes
*
struct ls_ucode_img_desc ucode_desc;
u8 *ucode_data;
u32 ucode_size;
+ u32 ucode_off;
u8 *sig;
u32 sig_size;
int acr_ls_ucode_load_fecs(const struct nvkm_subdev *, struct ls_ucode_img *);
int acr_ls_ucode_load_gpccs(const struct nvkm_subdev *, struct ls_ucode_img *);
-
+int acr_ls_ucode_load_pmu(const struct nvkm_subdev *, struct ls_ucode_img *);
+void acr_ls_pmu_post_run(const struct nvkm_acr *, const struct nvkm_secboot *);
+int acr_ls_ucode_load_sec2(const struct nvkm_subdev *, struct ls_ucode_img *);
+void acr_ls_sec2_post_run(const struct nvkm_acr *, const struct nvkm_secboot *);
#endif
--- /dev/null
+/*
+ * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+
+#include "ls_ucode.h"
+#include "acr.h"
+
+#include <core/firmware.h>
+#include <core/msgqueue.h>
+#include <subdev/pmu.h>
+#include <engine/sec2.h>
+
+/**
+ * acr_ls_ucode_load_msgqueue - load and prepare a ucode img for a msgqueue fw
+ *
+ * Load the LS microcode, desc and signature and pack them into a single
+ * blob.
+ */
+static int
+acr_ls_ucode_load_msgqueue(const struct nvkm_subdev *subdev, const char *name,
+ struct ls_ucode_img *img)
+{
+ const struct firmware *image, *desc, *sig;
+ char f[64];
+ int ret;
+
+ snprintf(f, sizeof(f), "%s/image", name);
+ ret = nvkm_firmware_get(subdev->device, f, &image);
+ if (ret)
+ return ret;
+ img->ucode_data = kmemdup(image->data, image->size, GFP_KERNEL);
+ nvkm_firmware_put(image);
+ if (!img->ucode_data)
+ return -ENOMEM;
+
+ snprintf(f, sizeof(f), "%s/desc", name);
+ ret = nvkm_firmware_get(subdev->device, f, &desc);
+ if (ret)
+ return ret;
+ memcpy(&img->ucode_desc, desc->data, sizeof(img->ucode_desc));
+ img->ucode_size = ALIGN(img->ucode_desc.app_start_offset + img->ucode_desc.app_size, 256);
+ nvkm_firmware_put(desc);
+
+ snprintf(f, sizeof(f), "%s/sig", name);
+ ret = nvkm_firmware_get(subdev->device, f, &sig);
+ if (ret)
+ return ret;
+ img->sig_size = sig->size;
+ img->sig = kmemdup(sig->data, sig->size, GFP_KERNEL);
+ nvkm_firmware_put(sig);
+ if (!img->sig)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void
+acr_ls_msgqueue_post_run(struct nvkm_msgqueue *queue,
+ struct nvkm_falcon *falcon, u32 addr_args)
+{
+ u32 cmdline_size = NVKM_MSGQUEUE_CMDLINE_SIZE;
+ u8 buf[cmdline_size];
+
+ memset(buf, 0, cmdline_size);
+ nvkm_msgqueue_write_cmdline(queue, buf);
+ nvkm_falcon_load_dmem(falcon, buf, addr_args, cmdline_size, 0);
+ /* rearm the queue so it will wait for the init message */
+ nvkm_msgqueue_reinit(queue);
+}
+
+int
+acr_ls_ucode_load_pmu(const struct nvkm_subdev *subdev,
+ struct ls_ucode_img *img)
+{
+ struct nvkm_pmu *pmu = subdev->device->pmu;
+ int ret;
+
+ ret = acr_ls_ucode_load_msgqueue(subdev, "pmu", img);
+ if (ret)
+ return ret;
+
+ /* Allocate the PMU queue corresponding to the FW version */
+ ret = nvkm_msgqueue_new(img->ucode_desc.app_version, pmu->falcon,
+ &pmu->queue);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+void
+acr_ls_pmu_post_run(const struct nvkm_acr *acr, const struct nvkm_secboot *sb)
+{
+ struct nvkm_device *device = sb->subdev.device;
+ struct nvkm_pmu *pmu = device->pmu;
+ u32 addr_args = pmu->falcon->data.limit - NVKM_MSGQUEUE_CMDLINE_SIZE;
+
+ acr_ls_msgqueue_post_run(pmu->queue, pmu->falcon, addr_args);
+}
+
+int
+acr_ls_ucode_load_sec2(const struct nvkm_subdev *subdev,
+ struct ls_ucode_img *img)
+{
+ struct nvkm_sec2 *sec = subdev->device->sec2;
+ int ret;
+
+ ret = acr_ls_ucode_load_msgqueue(subdev, "sec2", img);
+ if (ret)
+ return ret;
+
+ /* Allocate the PMU queue corresponding to the FW version */
+ ret = nvkm_msgqueue_new(img->ucode_desc.app_version, sec->falcon,
+ &sec->queue);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+void
+acr_ls_sec2_post_run(const struct nvkm_acr *acr, const struct nvkm_secboot *sb)
+{
+ struct nvkm_device *device = sb->subdev.device;
+ struct nvkm_sec2 *sec = device->sec2;
+ /* on SEC arguments are always at the beginning of EMEM */
+ u32 addr_args = 0x01000000;
+
+ acr_ls_msgqueue_post_run(sec->queue, sec->falcon, addr_args);
+}
int (*oneinit)(struct nvkm_secboot *);
int (*fini)(struct nvkm_secboot *, bool suspend);
void *(*dtor)(struct nvkm_secboot *);
- int (*run_blob)(struct nvkm_secboot *, struct nvkm_gpuobj *);
+ int (*run_blob)(struct nvkm_secboot *, struct nvkm_gpuobj *,
+ struct nvkm_falcon *);
};
-extern const char *nvkm_secboot_falcon_name[];
-
int nvkm_secboot_ctor(const struct nvkm_secboot_func *, struct nvkm_acr *,
struct nvkm_device *, int, struct nvkm_secboot *);
int nvkm_secboot_falcon_reset(struct nvkm_secboot *);
case 0x0000000a: A_(MSVLD ); break;
case 0x0000000b: A_(MSENC ); break;
case 0x0000000c: A_(VIC ); break;
- case 0x0000000d: A_(SEC ); break;
+ case 0x0000000d: A_(SEC2 ); break;
case 0x0000000e: B_(NVENC ); break;
case 0x0000000f: A_(NVENC1); break;
case 0x00000010: A_(NVDEC ); break;
total_bytes = dsi->update_bytes;
- printk(KERN_INFO "DSI(%s): %u us + %u us = %u us (%uHz), "
- "%u bytes, %u kbytes/sec\n",
- name,
- setup_us,
- trans_us,
- total_us,
- 1000*1000 / total_us,
- total_bytes,
- total_bytes * 1000 / total_us);
+ pr_info("DSI(%s): %u us + %u us = %u us (%uHz), %u bytes, %u kbytes/sec\n",
+ name,
+ setup_us,
+ trans_us,
+ total_us,
+ 1000 * 1000 / total_us,
+ total_bytes,
+ total_bytes * 1000 / total_us);
}
#else
static inline void dsi_perf_mark_setup(struct platform_device *dsidev)
dss.lcd_clk_source[1] = DSS_CLK_SRC_FCK;
rev = dss_read_reg(DSS_REVISION);
- printk(KERN_INFO "OMAP DSS rev %d.%d\n",
- FLD_GET(rev, 7, 4), FLD_GET(rev, 3, 0));
+ pr_info("OMAP DSS rev %d.%d\n", FLD_GET(rev, 7, 4), FLD_GET(rev, 3, 0));
dss_runtime_put();
#ifdef DSS_SUBSYS_NAME
#define DSSERR(format, ...) \
- printk(KERN_ERR "omapdss " DSS_SUBSYS_NAME " error: " format, \
- ## __VA_ARGS__)
+ pr_err("omapdss " DSS_SUBSYS_NAME " error: " format, ##__VA_ARGS__)
#else
#define DSSERR(format, ...) \
- printk(KERN_ERR "omapdss error: " format, ## __VA_ARGS__)
+ pr_err("omapdss error: " format, ##__VA_ARGS__)
#endif
#ifdef DSS_SUBSYS_NAME
#define DSSINFO(format, ...) \
- printk(KERN_INFO "omapdss " DSS_SUBSYS_NAME ": " format, \
- ## __VA_ARGS__)
+ pr_info("omapdss " DSS_SUBSYS_NAME ": " format, ##__VA_ARGS__)
#else
#define DSSINFO(format, ...) \
- printk(KERN_INFO "omapdss: " format, ## __VA_ARGS__)
+ pr_info("omapdss: " format, ## __VA_ARGS__)
#endif
#ifdef DSS_SUBSYS_NAME
#define DSSWARN(format, ...) \
- printk(KERN_WARNING "omapdss " DSS_SUBSYS_NAME ": " format, \
- ## __VA_ARGS__)
+ pr_warn("omapdss " DSS_SUBSYS_NAME ": " format, ##__VA_ARGS__)
#else
#define DSSWARN(format, ...) \
- printk(KERN_WARNING "omapdss: " format, ## __VA_ARGS__)
+ pr_warn("omapdss: " format, ##__VA_ARGS__)
#endif
/* OMAP TRM gives bitfields as start:end, where start is the higher bit
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
.atomic_set_property = omap_crtc_atomic_set_property,
.atomic_get_property = omap_crtc_atomic_get_property,
+ .enable_vblank = omap_irq_enable_vblank,
+ .disable_vblank = omap_irq_disable_vblank,
};
static const struct drm_crtc_helper_funcs omap_crtc_helper_funcs = {
DRIVER_ATOMIC,
.open = dev_open,
.lastclose = dev_lastclose,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = omap_irq_enable_vblank,
- .disable_vblank = omap_irq_disable_vblank,
#ifdef CONFIG_DEBUG_FS
.debugfs_init = omap_debugfs_init,
#endif
int omap_gem_resume(struct device *dev);
#endif
-int omap_irq_enable_vblank(struct drm_device *dev, unsigned int pipe);
-void omap_irq_disable_vblank(struct drm_device *dev, unsigned int pipe);
+int omap_irq_enable_vblank(struct drm_crtc *crtc);
+void omap_irq_disable_vblank(struct drm_crtc *crtc);
void omap_drm_irq_uninstall(struct drm_device *dev);
int omap_drm_irq_install(struct drm_device *dev);
fail:
if (ret) {
-
- drm_fb_helper_release_fbi(helper);
-
if (fb)
drm_framebuffer_remove(fb);
}
DBG();
drm_fb_helper_unregister_fbi(helper);
- drm_fb_helper_release_fbi(helper);
drm_fb_helper_fini(helper);
/* macro for sync debug.. */
#define SYNCDBG 0
-#define SYNC(fmt, ...) do { if (SYNCDBG) \
- printk(KERN_ERR "%s:%d: "fmt"\n", \
- __func__, __LINE__, ##__VA_ARGS__); \
+#define SYNC(fmt, ...) do { if (SYNCDBG) \
+ pr_err("%s:%d: " fmt "\n", __func__, __LINE__, ##__VA_ARGS__); \
} while (0)
* Zero on success, appropriate errno if the given @crtc's vblank
* interrupt cannot be enabled.
*/
-int omap_irq_enable_vblank(struct drm_device *dev, unsigned int pipe)
+int omap_irq_enable_vblank(struct drm_crtc *crtc)
{
+ struct drm_device *dev = crtc->dev;
struct omap_drm_private *priv = dev->dev_private;
- struct drm_crtc *crtc = priv->crtcs[pipe];
unsigned long flags;
+ enum omap_channel channel = omap_crtc_channel(crtc);
- DBG("dev=%p, crtc=%u", dev, pipe);
+ DBG("dev=%p, crtc=%u", dev, channel);
spin_lock_irqsave(&priv->wait_lock, flags);
- priv->irq_mask |= dispc_mgr_get_vsync_irq(omap_crtc_channel(crtc));
+ priv->irq_mask |= dispc_mgr_get_vsync_irq(channel);
omap_irq_update(dev);
spin_unlock_irqrestore(&priv->wait_lock, flags);
* a hardware vblank counter, this routine should be a no-op, since
* interrupts will have to stay on to keep the count accurate.
*/
-void omap_irq_disable_vblank(struct drm_device *dev, unsigned int pipe)
+void omap_irq_disable_vblank(struct drm_crtc *crtc)
{
+ struct drm_device *dev = crtc->dev;
struct omap_drm_private *priv = dev->dev_private;
- struct drm_crtc *crtc = priv->crtcs[pipe];
unsigned long flags;
+ enum omap_channel channel = omap_crtc_channel(crtc);
- DBG("dev=%p, crtc=%u", dev, pipe);
+ DBG("dev=%p, crtc=%u", dev, channel);
spin_lock_irqsave(&priv->wait_lock, flags);
- priv->irq_mask &= ~dispc_mgr_get_vsync_irq(omap_crtc_channel(crtc));
+ priv->irq_mask &= ~dispc_mgr_get_vsync_irq(channel);
omap_irq_update(dev);
spin_unlock_irqrestore(&priv->wait_lock, flags);
}
return 0;
}
-void
-qxl_debugfs_takedown(struct drm_minor *minor)
-{
-#if defined(CONFIG_DEBUG_FS)
- drm_debugfs_remove_files(qxl_debugfs_list, QXL_DEBUGFS_ENTRIES,
- minor);
-#endif
-}
-
int qxl_debugfs_add_files(struct qxl_device *qdev,
struct drm_info_list *files,
unsigned nfiles)
#endif
return 0;
}
-
-void qxl_debugfs_remove_files(struct qxl_device *qdev)
-{
-#if defined(CONFIG_DEBUG_FS)
- unsigned i;
-
- for (i = 0; i < qdev->debugfs_count; i++) {
- drm_debugfs_remove_files(qdev->debugfs[i].files,
- qdev->debugfs[i].num_files,
- qdev->ddev.primary);
- }
-#endif
-}
#include "qxl_object.h"
#include "drm_crtc_helper.h"
#include <drm/drm_plane_helper.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_atomic.h>
static bool qxl_head_enabled(struct qxl_head *head)
{
qdev->rom->client_monitors_config_crc);
return MONITORS_CONFIG_BAD_CRC;
}
+ if (!num_monitors) {
+ DRM_DEBUG_KMS("no client monitors configured\n");
+ return status;
+ }
if (num_monitors > qdev->monitors_config->max_allowed) {
DRM_DEBUG_KMS("client monitors list will be truncated: %d < %d\n",
qdev->monitors_config->max_allowed, num_monitors);
void qxl_display_read_client_monitors_config(struct qxl_device *qdev)
{
-
struct drm_device *dev = &qdev->ddev;
- int status;
+ int status, retries;
- status = qxl_display_copy_rom_client_monitors_config(qdev);
- while (status == MONITORS_CONFIG_BAD_CRC) {
- qxl_io_log(qdev, "failed crc check for client_monitors_config,"
- " retrying\n");
+ for (retries = 0; retries < 10; retries++) {
status = qxl_display_copy_rom_client_monitors_config(qdev);
+ if (status != MONITORS_CONFIG_BAD_CRC)
+ break;
+ udelay(5);
+ }
+ if (status == MONITORS_CONFIG_BAD_CRC) {
+ qxl_io_log(qdev, "config: bad crc\n");
+ DRM_DEBUG_KMS("ignoring client monitors config: bad crc");
+ return;
}
if (status == MONITORS_CONFIG_UNCHANGED) {
- qxl_io_log(qdev, "config unchanged\n");
- DRM_DEBUG("ignoring unchanged client monitors config");
+ qxl_io_log(qdev, "config: unchanged\n");
+ DRM_DEBUG_KMS("ignoring client monitors config: unchanged");
return;
}
struct drm_display_mode *mode = NULL;
struct qxl_head *head;
+ if (!qdev->monitors_config)
+ return 0;
+ if (h >= qdev->monitors_config->max_allowed)
+ return 0;
if (!qdev->client_monitors_config)
return 0;
+ if (h >= qdev->client_monitors_config->count)
+ return 0;
+
head = &qdev->client_monitors_config->heads[h];
+ DRM_DEBUG_KMS("head %d is %dx%d\n", h, head->width, head->height);
mode = drm_cvt_mode(dev, head->width, head->height, 60, false, false,
false);
return i - 1;
}
-static void qxl_crtc_destroy(struct drm_crtc *crtc)
-{
- struct qxl_crtc *qxl_crtc = to_qxl_crtc(crtc);
-
- drm_crtc_cleanup(crtc);
- qxl_bo_unref(&qxl_crtc->cursor_bo);
- kfree(qxl_crtc);
-}
-
-static int qxl_crtc_page_flip(struct drm_crtc *crtc,
- struct drm_framebuffer *fb,
- struct drm_pending_vblank_event *event,
- uint32_t page_flip_flags)
+static void qxl_crtc_atomic_flush(struct drm_crtc *crtc,
+ struct drm_crtc_state *old_crtc_state)
{
struct drm_device *dev = crtc->dev;
- struct qxl_device *qdev = dev->dev_private;
- struct qxl_framebuffer *qfb_src = to_qxl_framebuffer(fb);
- struct qxl_framebuffer *qfb_old = to_qxl_framebuffer(crtc->primary->fb);
- struct qxl_bo *bo_old = gem_to_qxl_bo(qfb_old->obj);
- struct qxl_bo *bo = gem_to_qxl_bo(qfb_src->obj);
+ struct drm_pending_vblank_event *event;
unsigned long flags;
- struct drm_clip_rect norect = {
- .x1 = 0,
- .y1 = 0,
- .x2 = fb->width,
- .y2 = fb->height
- };
- int inc = 1;
- int one_clip_rect = 1;
- int ret = 0;
- crtc->primary->fb = fb;
- bo_old->is_primary = false;
- bo->is_primary = true;
+ if (crtc->state && crtc->state->event) {
+ event = crtc->state->event;
+ crtc->state->event = NULL;
- ret = qxl_bo_reserve(bo, false);
- if (ret)
- return ret;
- ret = qxl_bo_pin(bo, bo->type, NULL);
- qxl_bo_unreserve(bo);
- if (ret)
- return ret;
-
- qxl_draw_dirty_fb(qdev, qfb_src, bo, 0, 0,
- &norect, one_clip_rect, inc);
-
- drm_crtc_vblank_get(crtc);
-
- if (event) {
spin_lock_irqsave(&dev->event_lock, flags);
drm_crtc_send_vblank_event(crtc, event);
spin_unlock_irqrestore(&dev->event_lock, flags);
}
- drm_crtc_vblank_put(crtc);
-
- ret = qxl_bo_reserve(bo, false);
- if (!ret) {
- qxl_bo_unpin(bo);
- qxl_bo_unreserve(bo);
- }
-
- return 0;
}
-static int
-qxl_hide_cursor(struct qxl_device *qdev)
-{
- struct qxl_release *release;
- struct qxl_cursor_cmd *cmd;
- int ret;
-
- ret = qxl_alloc_release_reserved(qdev, sizeof(*cmd), QXL_RELEASE_CURSOR_CMD,
- &release, NULL);
- if (ret)
- return ret;
-
- ret = qxl_release_reserve_list(release, true);
- if (ret) {
- qxl_release_free(qdev, release);
- return ret;
- }
-
- cmd = (struct qxl_cursor_cmd *)qxl_release_map(qdev, release);
- cmd->type = QXL_CURSOR_HIDE;
- qxl_release_unmap(qdev, release, &cmd->release_info);
-
- qxl_push_cursor_ring_release(qdev, release, QXL_CMD_CURSOR, false);
- qxl_release_fence_buffer_objects(release);
- return 0;
-}
-
-static int qxl_crtc_apply_cursor(struct drm_crtc *crtc)
-{
- struct qxl_crtc *qcrtc = to_qxl_crtc(crtc);
- struct drm_device *dev = crtc->dev;
- struct qxl_device *qdev = dev->dev_private;
- struct qxl_cursor_cmd *cmd;
- struct qxl_release *release;
- int ret = 0;
-
- if (!qcrtc->cursor_bo)
- return 0;
-
- ret = qxl_alloc_release_reserved(qdev, sizeof(*cmd),
- QXL_RELEASE_CURSOR_CMD,
- &release, NULL);
- if (ret)
- return ret;
-
- ret = qxl_release_list_add(release, qcrtc->cursor_bo);
- if (ret)
- goto out_free_release;
-
- ret = qxl_release_reserve_list(release, false);
- if (ret)
- goto out_free_release;
-
- cmd = (struct qxl_cursor_cmd *)qxl_release_map(qdev, release);
- cmd->type = QXL_CURSOR_SET;
- cmd->u.set.position.x = qcrtc->cur_x + qcrtc->hot_spot_x;
- cmd->u.set.position.y = qcrtc->cur_y + qcrtc->hot_spot_y;
-
- cmd->u.set.shape = qxl_bo_physical_address(qdev, qcrtc->cursor_bo, 0);
-
- cmd->u.set.visible = 1;
- qxl_release_unmap(qdev, release, &cmd->release_info);
-
- qxl_push_cursor_ring_release(qdev, release, QXL_CMD_CURSOR, false);
- qxl_release_fence_buffer_objects(release);
-
- return ret;
-
-out_free_release:
- qxl_release_free(qdev, release);
- return ret;
-}
-
-static int qxl_crtc_cursor_set2(struct drm_crtc *crtc,
- struct drm_file *file_priv,
- uint32_t handle,
- uint32_t width,
- uint32_t height, int32_t hot_x, int32_t hot_y)
-{
- struct drm_device *dev = crtc->dev;
- struct qxl_device *qdev = dev->dev_private;
- struct qxl_crtc *qcrtc = to_qxl_crtc(crtc);
- struct drm_gem_object *obj;
- struct qxl_cursor *cursor;
- struct qxl_cursor_cmd *cmd;
- struct qxl_bo *cursor_bo, *user_bo;
- struct qxl_release *release;
- void *user_ptr;
-
- int size = 64*64*4;
- int ret = 0;
- if (!handle)
- return qxl_hide_cursor(qdev);
-
- obj = drm_gem_object_lookup(file_priv, handle);
- if (!obj) {
- DRM_ERROR("cannot find cursor object\n");
- return -ENOENT;
- }
-
- user_bo = gem_to_qxl_bo(obj);
-
- ret = qxl_bo_reserve(user_bo, false);
- if (ret)
- goto out_unref;
-
- ret = qxl_bo_pin(user_bo, QXL_GEM_DOMAIN_CPU, NULL);
- qxl_bo_unreserve(user_bo);
- if (ret)
- goto out_unref;
-
- ret = qxl_bo_kmap(user_bo, &user_ptr);
- if (ret)
- goto out_unpin;
-
- ret = qxl_alloc_release_reserved(qdev, sizeof(*cmd),
- QXL_RELEASE_CURSOR_CMD,
- &release, NULL);
- if (ret)
- goto out_kunmap;
-
- ret = qxl_alloc_bo_reserved(qdev, release, sizeof(struct qxl_cursor) + size,
- &cursor_bo);
- if (ret)
- goto out_free_release;
-
- ret = qxl_release_reserve_list(release, false);
- if (ret)
- goto out_free_bo;
-
- ret = qxl_bo_kmap(cursor_bo, (void **)&cursor);
- if (ret)
- goto out_backoff;
-
- cursor->header.unique = 0;
- cursor->header.type = SPICE_CURSOR_TYPE_ALPHA;
- cursor->header.width = 64;
- cursor->header.height = 64;
- cursor->header.hot_spot_x = hot_x;
- cursor->header.hot_spot_y = hot_y;
- cursor->data_size = size;
- cursor->chunk.next_chunk = 0;
- cursor->chunk.prev_chunk = 0;
- cursor->chunk.data_size = size;
-
- memcpy(cursor->chunk.data, user_ptr, size);
-
- qxl_bo_kunmap(cursor_bo);
-
- qxl_bo_kunmap(user_bo);
-
- qcrtc->cur_x += qcrtc->hot_spot_x - hot_x;
- qcrtc->cur_y += qcrtc->hot_spot_y - hot_y;
- qcrtc->hot_spot_x = hot_x;
- qcrtc->hot_spot_y = hot_y;
-
- cmd = (struct qxl_cursor_cmd *)qxl_release_map(qdev, release);
- cmd->type = QXL_CURSOR_SET;
- cmd->u.set.position.x = qcrtc->cur_x + qcrtc->hot_spot_x;
- cmd->u.set.position.y = qcrtc->cur_y + qcrtc->hot_spot_y;
-
- cmd->u.set.shape = qxl_bo_physical_address(qdev, cursor_bo, 0);
-
- cmd->u.set.visible = 1;
- qxl_release_unmap(qdev, release, &cmd->release_info);
-
- qxl_push_cursor_ring_release(qdev, release, QXL_CMD_CURSOR, false);
- qxl_release_fence_buffer_objects(release);
-
- /* finish with the userspace bo */
- ret = qxl_bo_reserve(user_bo, false);
- if (!ret) {
- qxl_bo_unpin(user_bo);
- qxl_bo_unreserve(user_bo);
- }
- drm_gem_object_unreference_unlocked(obj);
-
- qxl_bo_unref (&qcrtc->cursor_bo);
- qcrtc->cursor_bo = cursor_bo;
-
- return ret;
-
-out_backoff:
- qxl_release_backoff_reserve_list(release);
-out_free_bo:
- qxl_bo_unref(&cursor_bo);
-out_free_release:
- qxl_release_free(qdev, release);
-out_kunmap:
- qxl_bo_kunmap(user_bo);
-out_unpin:
- qxl_bo_unpin(user_bo);
-out_unref:
- drm_gem_object_unreference_unlocked(obj);
- return ret;
-}
-
-static int qxl_crtc_cursor_move(struct drm_crtc *crtc,
- int x, int y)
+static void qxl_crtc_destroy(struct drm_crtc *crtc)
{
- struct drm_device *dev = crtc->dev;
- struct qxl_device *qdev = dev->dev_private;
- struct qxl_crtc *qcrtc = to_qxl_crtc(crtc);
- struct qxl_release *release;
- struct qxl_cursor_cmd *cmd;
- int ret;
-
- ret = qxl_alloc_release_reserved(qdev, sizeof(*cmd), QXL_RELEASE_CURSOR_CMD,
- &release, NULL);
- if (ret)
- return ret;
-
- ret = qxl_release_reserve_list(release, true);
- if (ret) {
- qxl_release_free(qdev, release);
- return ret;
- }
-
- qcrtc->cur_x = x;
- qcrtc->cur_y = y;
-
- cmd = (struct qxl_cursor_cmd *)qxl_release_map(qdev, release);
- cmd->type = QXL_CURSOR_MOVE;
- cmd->u.position.x = qcrtc->cur_x + qcrtc->hot_spot_x;
- cmd->u.position.y = qcrtc->cur_y + qcrtc->hot_spot_y;
- qxl_release_unmap(qdev, release, &cmd->release_info);
-
- qxl_push_cursor_ring_release(qdev, release, QXL_CMD_CURSOR, false);
- qxl_release_fence_buffer_objects(release);
+ struct qxl_crtc *qxl_crtc = to_qxl_crtc(crtc);
- return 0;
+ drm_crtc_cleanup(crtc);
+ kfree(qxl_crtc);
}
-
static const struct drm_crtc_funcs qxl_crtc_funcs = {
- .cursor_set2 = qxl_crtc_cursor_set2,
- .cursor_move = qxl_crtc_cursor_move,
- .set_config = drm_crtc_helper_set_config,
+ .set_config = drm_atomic_helper_set_config,
.destroy = qxl_crtc_destroy,
- .page_flip = qxl_crtc_page_flip,
+ .page_flip = drm_atomic_helper_page_flip,
+ .reset = drm_atomic_helper_crtc_reset,
+ .atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
};
void qxl_user_framebuffer_destroy(struct drm_framebuffer *fb)
}
-static int qxl_crtc_mode_set(struct drm_crtc *crtc,
- struct drm_display_mode *mode,
- struct drm_display_mode *adjusted_mode,
- int x, int y,
- struct drm_framebuffer *old_fb)
+void qxl_mode_set_nofb(struct drm_crtc *crtc)
{
- struct drm_device *dev = crtc->dev;
- struct qxl_device *qdev = dev->dev_private;
- struct qxl_framebuffer *qfb;
- struct qxl_bo *bo, *old_bo = NULL;
+ struct qxl_device *qdev = crtc->dev->dev_private;
struct qxl_crtc *qcrtc = to_qxl_crtc(crtc);
- bool recreate_primary = false;
- int ret;
- int surf_id;
- if (!crtc->primary->fb) {
- DRM_DEBUG_KMS("No FB bound\n");
+ struct drm_display_mode *mode = &crtc->mode;
+
+ DRM_DEBUG("Mode set (%d,%d)\n",
+ mode->hdisplay, mode->vdisplay);
+
+ qxl_monitors_config_set(qdev, qcrtc->index, 0, 0,
+ mode->hdisplay, mode->vdisplay, 0);
+
+}
+
+static void qxl_crtc_commit(struct drm_crtc *crtc)
+{
+ DRM_DEBUG("\n");
+}
+
+static void qxl_crtc_disable(struct drm_crtc *crtc)
+{
+ struct qxl_crtc *qcrtc = to_qxl_crtc(crtc);
+ struct qxl_device *qdev = crtc->dev->dev_private;
+
+ qxl_monitors_config_set(qdev, qcrtc->index, 0, 0, 0, 0, 0);
+
+ qxl_send_monitors_config(qdev);
+}
+
+static const struct drm_crtc_helper_funcs qxl_crtc_helper_funcs = {
+ .dpms = qxl_crtc_dpms,
+ .disable = qxl_crtc_disable,
+ .mode_fixup = qxl_crtc_mode_fixup,
+ .mode_set_nofb = qxl_mode_set_nofb,
+ .commit = qxl_crtc_commit,
+ .atomic_flush = qxl_crtc_atomic_flush,
+};
+
+int qxl_primary_atomic_check(struct drm_plane *plane,
+ struct drm_plane_state *state)
+{
+ struct qxl_device *qdev = plane->dev->dev_private;
+ struct qxl_framebuffer *qfb;
+ struct qxl_bo *bo;
+
+ if (!state->crtc || !state->fb)
return 0;
- }
- if (old_fb) {
- qfb = to_qxl_framebuffer(old_fb);
- old_bo = gem_to_qxl_bo(qfb->obj);
- }
- qfb = to_qxl_framebuffer(crtc->primary->fb);
+ qfb = to_qxl_framebuffer(state->fb);
bo = gem_to_qxl_bo(qfb->obj);
- DRM_DEBUG("+%d+%d (%d,%d) => (%d,%d)\n",
- x, y,
- mode->hdisplay, mode->vdisplay,
- adjusted_mode->hdisplay,
- adjusted_mode->vdisplay);
-
- if (bo->is_primary == false)
- recreate_primary = true;
if (bo->surf.stride * bo->surf.height > qdev->vram_size) {
DRM_ERROR("Mode doesn't fit in vram size (vgamem)");
return -EINVAL;
- }
-
- ret = qxl_bo_reserve(bo, false);
- if (ret != 0)
- return ret;
- ret = qxl_bo_pin(bo, bo->type, NULL);
- if (ret != 0) {
- qxl_bo_unreserve(bo);
- return -EINVAL;
}
- qxl_bo_unreserve(bo);
- if (recreate_primary) {
- qxl_io_destroy_primary(qdev);
+
+ return 0;
+}
+
+static void qxl_primary_atomic_update(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
+{
+ struct qxl_device *qdev = plane->dev->dev_private;
+ struct qxl_framebuffer *qfb =
+ to_qxl_framebuffer(plane->state->fb);
+ struct qxl_framebuffer *qfb_old;
+ struct qxl_bo *bo = gem_to_qxl_bo(qfb->obj);
+ struct qxl_bo *bo_old;
+ struct drm_clip_rect norect = {
+ .x1 = 0,
+ .y1 = 0,
+ .x2 = qfb->base.width,
+ .y2 = qfb->base.height
+ };
+
+ if (!old_state->fb) {
qxl_io_log(qdev,
- "recreate primary: %dx%d,%d,%d\n",
+ "create primary fb: %dx%d,%d,%d\n",
bo->surf.width, bo->surf.height,
bo->surf.stride, bo->surf.format);
+
qxl_io_create_primary(qdev, 0, bo);
bo->is_primary = true;
+ return;
- ret = qxl_crtc_apply_cursor(crtc);
- if (ret) {
- DRM_ERROR("could not set cursor after modeset");
- ret = 0;
- }
- }
-
- if (bo->is_primary) {
- DRM_DEBUG_KMS("setting surface_id to 0 for primary surface %d on crtc %d\n", bo->surface_id, qcrtc->index);
- surf_id = 0;
} else {
- surf_id = bo->surface_id;
+ qfb_old = to_qxl_framebuffer(old_state->fb);
+ bo_old = gem_to_qxl_bo(qfb_old->obj);
+ bo_old->is_primary = false;
}
- if (old_bo && old_bo != bo) {
- old_bo->is_primary = false;
- ret = qxl_bo_reserve(old_bo, false);
- qxl_bo_unpin(old_bo);
- qxl_bo_unreserve(old_bo);
+ bo->is_primary = true;
+ qxl_draw_dirty_fb(qdev, qfb, bo, 0, 0, &norect, 1, 1);
+}
+
+static void qxl_primary_atomic_disable(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
+{
+ struct qxl_device *qdev = plane->dev->dev_private;
+
+ if (old_state->fb)
+ { struct qxl_framebuffer *qfb =
+ to_qxl_framebuffer(old_state->fb);
+ struct qxl_bo *bo = gem_to_qxl_bo(qfb->obj);
+
+ qxl_io_destroy_primary(qdev);
+ bo->is_primary = false;
}
+}
- qxl_monitors_config_set(qdev, qcrtc->index, x, y,
- mode->hdisplay,
- mode->vdisplay, surf_id);
+int qxl_plane_atomic_check(struct drm_plane *plane,
+ struct drm_plane_state *state)
+{
return 0;
}
-static void qxl_crtc_prepare(struct drm_crtc *crtc)
+static void qxl_cursor_atomic_update(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
{
- DRM_DEBUG("current: %dx%d+%d+%d (%d).\n",
- crtc->mode.hdisplay, crtc->mode.vdisplay,
- crtc->x, crtc->y, crtc->enabled);
+ struct drm_device *dev = plane->dev;
+ struct qxl_device *qdev = dev->dev_private;
+ struct drm_framebuffer *fb = plane->state->fb;
+ struct qxl_release *release;
+ struct qxl_cursor_cmd *cmd;
+ struct qxl_cursor *cursor;
+ struct drm_gem_object *obj;
+ struct qxl_bo *cursor_bo, *user_bo = NULL;
+ int ret;
+ void *user_ptr;
+ int size = 64*64*4;
+
+ ret = qxl_alloc_release_reserved(qdev, sizeof(*cmd),
+ QXL_RELEASE_CURSOR_CMD,
+ &release, NULL);
+ if (ret)
+ return;
+
+ cmd = (struct qxl_cursor_cmd *) qxl_release_map(qdev, release);
+
+ if (fb != old_state->fb) {
+ obj = to_qxl_framebuffer(fb)->obj;
+ user_bo = gem_to_qxl_bo(obj);
+
+ /* pinning is done in the prepare/cleanup framevbuffer */
+ ret = qxl_bo_kmap(user_bo, &user_ptr);
+ if (ret)
+ goto out_free_release;
+
+ ret = qxl_alloc_bo_reserved(qdev, release,
+ sizeof(struct qxl_cursor) + size,
+ &cursor_bo);
+ if (ret)
+ goto out_kunmap;
+
+ ret = qxl_release_reserve_list(release, true);
+ if (ret)
+ goto out_free_bo;
+
+ ret = qxl_bo_kmap(cursor_bo, (void **)&cursor);
+ if (ret)
+ goto out_backoff;
+
+ cursor->header.unique = 0;
+ cursor->header.type = SPICE_CURSOR_TYPE_ALPHA;
+ cursor->header.width = 64;
+ cursor->header.height = 64;
+ cursor->header.hot_spot_x = fb->hot_x;
+ cursor->header.hot_spot_y = fb->hot_y;
+ cursor->data_size = size;
+ cursor->chunk.next_chunk = 0;
+ cursor->chunk.prev_chunk = 0;
+ cursor->chunk.data_size = size;
+ memcpy(cursor->chunk.data, user_ptr, size);
+ qxl_bo_kunmap(cursor_bo);
+ qxl_bo_kunmap(user_bo);
+
+ cmd->u.set.visible = 1;
+ cmd->u.set.shape = qxl_bo_physical_address(qdev,
+ cursor_bo, 0);
+ cmd->type = QXL_CURSOR_SET;
+ } else {
+
+ ret = qxl_release_reserve_list(release, true);
+ if (ret)
+ goto out_free_release;
+
+ cmd->type = QXL_CURSOR_MOVE;
+ }
+
+ cmd->u.position.x = plane->state->crtc_x + fb->hot_x;
+ cmd->u.position.y = plane->state->crtc_y + fb->hot_y;
+
+ qxl_release_unmap(qdev, release, &cmd->release_info);
+ qxl_push_cursor_ring_release(qdev, release, QXL_CMD_CURSOR, false);
+ qxl_release_fence_buffer_objects(release);
+
+ return;
+
+out_backoff:
+ qxl_release_backoff_reserve_list(release);
+out_free_bo:
+ qxl_bo_unref(&cursor_bo);
+out_kunmap:
+ qxl_bo_kunmap(user_bo);
+out_free_release:
+ qxl_release_free(qdev, release);
+ return;
+
}
-static void qxl_crtc_commit(struct drm_crtc *crtc)
+void qxl_cursor_atomic_disable(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
{
- DRM_DEBUG("\n");
+ struct qxl_device *qdev = plane->dev->dev_private;
+ struct qxl_release *release;
+ struct qxl_cursor_cmd *cmd;
+ int ret;
+
+ ret = qxl_alloc_release_reserved(qdev, sizeof(*cmd),
+ QXL_RELEASE_CURSOR_CMD,
+ &release, NULL);
+ if (ret)
+ return;
+
+ ret = qxl_release_reserve_list(release, true);
+ if (ret) {
+ qxl_release_free(qdev, release);
+ return;
+ }
+
+ cmd = (struct qxl_cursor_cmd *)qxl_release_map(qdev, release);
+ cmd->type = QXL_CURSOR_HIDE;
+ qxl_release_unmap(qdev, release, &cmd->release_info);
+
+ qxl_push_cursor_ring_release(qdev, release, QXL_CMD_CURSOR, false);
+ qxl_release_fence_buffer_objects(release);
}
-static void qxl_crtc_disable(struct drm_crtc *crtc)
+int qxl_plane_prepare_fb(struct drm_plane *plane,
+ struct drm_plane_state *new_state)
{
- struct qxl_crtc *qcrtc = to_qxl_crtc(crtc);
- struct drm_device *dev = crtc->dev;
- struct qxl_device *qdev = dev->dev_private;
- if (crtc->primary->fb) {
- struct qxl_framebuffer *qfb = to_qxl_framebuffer(crtc->primary->fb);
- struct qxl_bo *bo = gem_to_qxl_bo(qfb->obj);
- int ret;
- ret = qxl_bo_reserve(bo, false);
- qxl_bo_unpin(bo);
- qxl_bo_unreserve(bo);
- crtc->primary->fb = NULL;
- }
+ struct drm_gem_object *obj;
+ struct qxl_bo *user_bo;
+ int ret;
- qxl_monitors_config_set(qdev, qcrtc->index, 0, 0, 0, 0, 0);
+ if (!new_state->fb)
+ return 0;
- qxl_send_monitors_config(qdev);
+ obj = to_qxl_framebuffer(new_state->fb)->obj;
+ user_bo = gem_to_qxl_bo(obj);
+
+ ret = qxl_bo_pin(user_bo, QXL_GEM_DOMAIN_CPU, NULL);
+ if (ret)
+ return ret;
+
+ return 0;
}
-static const struct drm_crtc_helper_funcs qxl_crtc_helper_funcs = {
- .dpms = qxl_crtc_dpms,
- .disable = qxl_crtc_disable,
- .mode_fixup = qxl_crtc_mode_fixup,
- .mode_set = qxl_crtc_mode_set,
- .prepare = qxl_crtc_prepare,
- .commit = qxl_crtc_commit,
+static void qxl_plane_cleanup_fb(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
+{
+ struct drm_gem_object *obj;
+ struct qxl_bo *user_bo;
+
+ if (!plane->state->fb) {
+ /* we never executed prepare_fb, so there's nothing to
+ * unpin.
+ */
+ return;
+ }
+
+ obj = to_qxl_framebuffer(plane->state->fb)->obj;
+ user_bo = gem_to_qxl_bo(obj);
+ qxl_bo_unpin(user_bo);
+}
+
+static const uint32_t qxl_cursor_plane_formats[] = {
+ DRM_FORMAT_ARGB8888,
+};
+
+static const struct drm_plane_helper_funcs qxl_cursor_helper_funcs = {
+ .atomic_check = qxl_plane_atomic_check,
+ .atomic_update = qxl_cursor_atomic_update,
+ .atomic_disable = qxl_cursor_atomic_disable,
+ .prepare_fb = qxl_plane_prepare_fb,
+ .cleanup_fb = qxl_plane_cleanup_fb,
+};
+
+static const struct drm_plane_funcs qxl_cursor_plane_funcs = {
+ .update_plane = drm_atomic_helper_update_plane,
+ .disable_plane = drm_atomic_helper_disable_plane,
+ .destroy = drm_primary_helper_destroy,
+ .reset = drm_atomic_helper_plane_reset,
+ .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
+};
+
+static const uint32_t qxl_primary_plane_formats[] = {
+ DRM_FORMAT_XRGB8888,
+ DRM_FORMAT_ARGB8888,
+};
+
+static const struct drm_plane_helper_funcs primary_helper_funcs = {
+ .atomic_check = qxl_primary_atomic_check,
+ .atomic_update = qxl_primary_atomic_update,
+ .atomic_disable = qxl_primary_atomic_disable,
+ .prepare_fb = qxl_plane_prepare_fb,
+ .cleanup_fb = qxl_plane_cleanup_fb,
};
+static const struct drm_plane_funcs qxl_primary_plane_funcs = {
+ .update_plane = drm_atomic_helper_update_plane,
+ .disable_plane = drm_atomic_helper_disable_plane,
+ .destroy = drm_primary_helper_destroy,
+ .reset = drm_atomic_helper_plane_reset,
+ .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
+};
+
+static struct drm_plane *qxl_create_plane(struct qxl_device *qdev,
+ unsigned int possible_crtcs,
+ enum drm_plane_type type)
+{
+ const struct drm_plane_helper_funcs *helper_funcs = NULL;
+ struct drm_plane *plane;
+ const struct drm_plane_funcs *funcs;
+ const uint32_t *formats;
+ int num_formats;
+ int err;
+
+ if (type == DRM_PLANE_TYPE_PRIMARY) {
+ funcs = &qxl_primary_plane_funcs;
+ formats = qxl_primary_plane_formats;
+ num_formats = ARRAY_SIZE(qxl_primary_plane_formats);
+ helper_funcs = &primary_helper_funcs;
+ } else if (type == DRM_PLANE_TYPE_CURSOR) {
+ funcs = &qxl_cursor_plane_funcs;
+ formats = qxl_cursor_plane_formats;
+ helper_funcs = &qxl_cursor_helper_funcs;
+ num_formats = ARRAY_SIZE(qxl_cursor_plane_formats);
+ } else {
+ return ERR_PTR(-EINVAL);
+ }
+
+ plane = kzalloc(sizeof(*plane), GFP_KERNEL);
+ if (!plane)
+ return ERR_PTR(-ENOMEM);
+
+ err = drm_universal_plane_init(&qdev->ddev, plane, possible_crtcs,
+ funcs, formats, num_formats,
+ type, NULL);
+ if (err)
+ goto free_plane;
+
+ drm_plane_helper_add(plane, helper_funcs);
+
+ return plane;
+
+free_plane:
+ kfree(plane);
+ return ERR_PTR(-EINVAL);
+}
+
static int qdev_crtc_init(struct drm_device *dev, int crtc_id)
{
struct qxl_crtc *qxl_crtc;
+ struct drm_plane *primary, *cursor;
+ struct qxl_device *qdev = dev->dev_private;
+ int r;
qxl_crtc = kzalloc(sizeof(struct qxl_crtc), GFP_KERNEL);
if (!qxl_crtc)
return -ENOMEM;
- drm_crtc_init(dev, &qxl_crtc->base, &qxl_crtc_funcs);
+ primary = qxl_create_plane(qdev, 1 << crtc_id, DRM_PLANE_TYPE_PRIMARY);
+ if (IS_ERR(primary)) {
+ r = -ENOMEM;
+ goto free_mem;
+ }
+
+ cursor = qxl_create_plane(qdev, 1 << crtc_id, DRM_PLANE_TYPE_CURSOR);
+ if (IS_ERR(cursor)) {
+ r = -ENOMEM;
+ goto clean_primary;
+ }
+
+ r = drm_crtc_init_with_planes(dev, &qxl_crtc->base, primary, cursor,
+ &qxl_crtc_funcs, NULL);
+ if (r)
+ goto clean_cursor;
+
qxl_crtc->index = crtc_id;
drm_crtc_helper_add(&qxl_crtc->base, &qxl_crtc_helper_funcs);
return 0;
+
+clean_cursor:
+ drm_plane_cleanup(cursor);
+ kfree(cursor);
+clean_primary:
+ drm_plane_cleanup(primary);
+ kfree(primary);
+free_mem:
+ kfree(qxl_crtc);
+ return r;
}
static void qxl_enc_dpms(struct drm_encoder *encoder, int mode)
static int qxl_conn_get_modes(struct drm_connector *connector)
{
- int ret = 0;
- struct qxl_device *qdev = connector->dev->dev_private;
unsigned pwidth = 1024;
unsigned pheight = 768;
+ int ret = 0;
- DRM_DEBUG_KMS("monitors_config=%p\n", qdev->monitors_config);
- /* TODO: what should we do here? only show the configured modes for the
- * device, or allow the full list, or both? */
- if (qdev->monitors_config && qdev->monitors_config->count) {
- ret = qxl_add_monitors_config_modes(connector, &pwidth, &pheight);
- if (ret < 0)
- return ret;
- }
+ ret = qxl_add_monitors_config_modes(connector, &pwidth, &pheight);
+ if (ret < 0)
+ return ret;
ret += qxl_add_common_modes(connector, pwidth, pheight);
return ret;
}
.fill_modes = drm_helper_probe_single_connector_modes,
.set_property = qxl_conn_set_property,
.destroy = qxl_conn_destroy,
+ .reset = drm_atomic_helper_connector_reset,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
};
static void qxl_enc_destroy(struct drm_encoder *encoder)
static const struct drm_mode_config_funcs qxl_mode_funcs = {
.fb_create = qxl_user_framebuffer_create,
+ .atomic_check = drm_atomic_helper_check,
+ .atomic_commit = drm_atomic_helper_commit,
};
int qxl_create_monitors_object(struct qxl_device *qdev)
}
qdev->monitors_config_bo = gem_to_qxl_bo(gobj);
- ret = qxl_bo_reserve(qdev->monitors_config_bo, false);
- if (ret)
- return ret;
-
ret = qxl_bo_pin(qdev->monitors_config_bo, QXL_GEM_DOMAIN_VRAM, NULL);
- if (ret) {
- qxl_bo_unreserve(qdev->monitors_config_bo);
+ if (ret)
return ret;
- }
-
- qxl_bo_unreserve(qdev->monitors_config_bo);
qxl_bo_kmap(qdev->monitors_config_bo, NULL);
qdev->ram_header->monitors_config = 0;
qxl_bo_kunmap(qdev->monitors_config_bo);
- ret = qxl_bo_reserve(qdev->monitors_config_bo, false);
+ ret = qxl_bo_unpin(qdev->monitors_config_bo);
if (ret)
return ret;
- qxl_bo_unpin(qdev->monitors_config_bo);
- qxl_bo_unreserve(qdev->monitors_config_bo);
-
qxl_bo_unref(&qdev->monitors_config_bo);
return 0;
}
qdev->ddev.mode_config.funcs = (void *)&qxl_mode_funcs;
/* modes will be validated against the framebuffer size */
- qdev->ddev.mode_config.min_width = 320;
- qdev->ddev.mode_config.min_height = 200;
+ qdev->ddev.mode_config.min_width = 0;
+ qdev->ddev.mode_config.min_height = 0;
qdev->ddev.mode_config.max_width = 8192;
qdev->ddev.mode_config.max_height = 8192;
qdev_output_init(&qdev->ddev, i);
}
+ qxl_display_read_client_monitors_config(qdev);
qdev->mode_info.mode_config_initialized = true;
+ drm_mode_config_reset(&qdev->ddev);
+
/* primary surface must be created by this point, to allow
* issuing command queue commands and having them read by
* spice server. */
if (ret)
goto free_dev;
- ret = qxl_device_init(qdev, &qxl_driver, pdev, ent->driver_data);
+ ret = qxl_device_init(qdev, &qxl_driver, pdev);
if (ret)
goto disable_pci;
- ret = drm_vblank_init(&qdev->ddev, 1);
- if (ret)
- goto unload;
-
ret = qxl_modeset_init(qdev);
if (ret)
- goto vblank_cleanup;
+ goto unload;
drm_kms_helper_poll_init(&qdev->ddev);
modeset_cleanup:
qxl_modeset_fini(qdev);
-vblank_cleanup:
- drm_vblank_cleanup(&qdev->ddev);
unload:
qxl_device_fini(qdev);
disable_pci:
return qxl_drm_resume(drm_dev, false);
}
-static u32 qxl_noop_get_vblank_counter(struct drm_device *dev,
- unsigned int pipe)
-{
- return 0;
-}
-
-static int qxl_noop_enable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- return 0;
-}
-
-static void qxl_noop_disable_vblank(struct drm_device *dev, unsigned int pipe)
-{
-}
-
static const struct dev_pm_ops qxl_pm_ops = {
.suspend = qxl_pm_suspend,
.resume = qxl_pm_resume,
static struct drm_driver qxl_driver = {
.driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_PRIME |
- DRIVER_HAVE_IRQ | DRIVER_IRQ_SHARED,
- .get_vblank_counter = qxl_noop_get_vblank_counter,
- .enable_vblank = qxl_noop_enable_vblank,
- .disable_vblank = qxl_noop_disable_vblank,
+ DRIVER_HAVE_IRQ | DRIVER_IRQ_SHARED |
+ DRIVER_ATOMIC,
.set_busid = drm_pci_set_busid,
.dumb_destroy = drm_gem_dumb_destroy,
#if defined(CONFIG_DEBUG_FS)
.debugfs_init = qxl_debugfs_init,
- .debugfs_cleanup = qxl_debugfs_takedown,
#endif
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
struct qxl_crtc {
struct drm_crtc base;
int index;
- int cur_x;
- int cur_y;
- int hot_spot_x;
- int hot_spot_y;
- struct qxl_bo *cursor_bo;
};
struct qxl_output {
};
struct qxl_mode_info {
- int num_modes;
- struct qxl_mode *modes;
bool mode_config_initialized;
/* pointer to fbdev info structure */
struct drm_info_list *files,
unsigned nfiles);
int qxl_debugfs_fence_init(struct qxl_device *rdev);
-void qxl_debugfs_remove_files(struct qxl_device *qdev);
struct qxl_device;
struct qxl_device {
struct drm_device ddev;
- unsigned long flags;
resource_size_t vram_base, vram_size;
resource_size_t surfaceram_base, surfaceram_size;
extern int qxl_max_ioctl;
int qxl_device_init(struct qxl_device *qdev, struct drm_driver *drv,
- struct pci_dev *pdev, unsigned long flags);
+ struct pci_dev *pdev);
void qxl_device_fini(struct qxl_device *qdev);
int qxl_modeset_init(struct qxl_device *qdev);
/* debugfs */
int qxl_debugfs_init(struct drm_minor *minor);
-void qxl_debugfs_takedown(struct drm_minor *minor);
int qxl_ttm_debugfs_init(struct qxl_device *qdev);
/* qxl_prime.c */
static void qxlfb_destroy_pinned_object(struct drm_gem_object *gobj)
{
struct qxl_bo *qbo = gem_to_qxl_bo(gobj);
- int ret;
- ret = qxl_bo_reserve(qbo, false);
- if (likely(ret == 0)) {
- qxl_bo_kunmap(qbo);
- qxl_bo_unpin(qbo);
- qxl_bo_unreserve(qbo);
- }
+ qxl_bo_kunmap(qbo);
+ qxl_bo_unpin(qbo);
+
drm_gem_object_unreference_unlocked(gobj);
}
qbo->surf.height = mode_cmd->height;
qbo->surf.stride = mode_cmd->pitches[0];
qbo->surf.format = SPICE_SURFACE_FMT_32_xRGB;
- ret = qxl_bo_reserve(qbo, false);
- if (unlikely(ret != 0))
- goto out_unref;
+
ret = qxl_bo_pin(qbo, QXL_GEM_DOMAIN_SURFACE, NULL);
if (ret) {
- qxl_bo_unreserve(qbo);
goto out_unref;
}
ret = qxl_bo_kmap(qbo, NULL);
- qxl_bo_unreserve(qbo); /* unreserve, will be mmaped */
+
if (ret)
goto out_unref;
if (info->screen_base == NULL) {
ret = -ENOSPC;
- goto out_destroy_fbi;
+ goto out_unref;
}
#ifdef CONFIG_DRM_FBDEV_EMULATION
fb->format->depth, fb->pitches[0], fb->width, fb->height);
return 0;
-out_destroy_fbi:
- drm_fb_helper_release_fbi(&qfbdev->helper);
out_unref:
if (qbo) {
- ret = qxl_bo_reserve(qbo, false);
- if (likely(ret == 0)) {
- qxl_bo_kunmap(qbo);
- qxl_bo_unpin(qbo);
- qxl_bo_unreserve(qbo);
- }
+ qxl_bo_kunmap(qbo);
+ qxl_bo_unpin(qbo);
}
if (fb && ret) {
drm_gem_object_unreference_unlocked(gobj);
struct qxl_framebuffer *qfb = &qfbdev->qfb;
drm_fb_helper_unregister_fbi(&qfbdev->helper);
- drm_fb_helper_release_fbi(&qfbdev->helper);
if (qfb->obj) {
qxlfb_destroy_pinned_object(qfb->obj);
int qxl_fbdev_init(struct qxl_device *qdev)
{
+ int ret = 0;
+
+#ifdef CONFIG_DRM_FBDEV_EMULATION
struct qxl_fbdev *qfbdev;
int bpp_sel = 32; /* TODO: parameter from somewhere? */
- int ret;
qfbdev = kzalloc(sizeof(struct qxl_fbdev), GFP_KERNEL);
if (!qfbdev)
drm_fb_helper_fini(&qfbdev->helper);
free:
kfree(qfbdev);
+#endif
+
return ret;
}
void qxl_fbdev_set_suspend(struct qxl_device *qdev, int state)
{
+ if (!qdev->mode_info.qfbdev)
+ return;
+
drm_fb_helper_set_suspend(&qdev->mode_info.qfbdev->helper, state);
}
int qxl_log_level;
-static void qxl_dump_mode(struct qxl_device *qdev, void *p)
-{
- struct qxl_mode *m = p;
- DRM_DEBUG_KMS("%d: %dx%d %d bits, stride %d, %dmm x %dmm, orientation %d\n",
- m->id, m->x_res, m->y_res, m->bits, m->stride, m->x_mili,
- m->y_mili, m->orientation);
-}
-
static bool qxl_check_device(struct qxl_device *qdev)
{
struct qxl_rom *rom = qdev->rom;
- int mode_offset;
- int i;
if (rom->magic != 0x4f525851) {
DRM_ERROR("bad rom signature %x\n", rom->magic);
DRM_INFO("Device Version %d.%d\n", rom->id, rom->update_id);
DRM_INFO("Compression level %d log level %d\n", rom->compression_level,
rom->log_level);
- DRM_INFO("Currently using mode #%d, list at 0x%x\n",
- rom->mode, rom->modes_offset);
DRM_INFO("%d io pages at offset 0x%x\n",
rom->num_io_pages, rom->pages_offset);
DRM_INFO("%d byte draw area at offset 0x%x\n",
qdev->vram_size = rom->surface0_area_size;
DRM_INFO("RAM header offset: 0x%x\n", rom->ram_header_offset);
-
- mode_offset = rom->modes_offset / 4;
- qdev->mode_info.num_modes = ((u32 *)rom)[mode_offset];
- DRM_INFO("rom modes offset 0x%x for %d modes\n", rom->modes_offset,
- qdev->mode_info.num_modes);
- qdev->mode_info.modes = (void *)((uint32_t *)rom + mode_offset + 1);
- for (i = 0; i < qdev->mode_info.num_modes; i++)
- qxl_dump_mode(qdev, qdev->mode_info.modes + i);
return true;
}
int qxl_device_init(struct qxl_device *qdev,
struct drm_driver *drv,
- struct pci_dev *pdev,
- unsigned long flags)
+ struct pci_dev *pdev)
{
int r, sb;
pci_set_drvdata(pdev, &qdev->ddev);
qdev->ddev.dev_private = qdev;
- qdev->flags = flags;
-
mutex_init(&qdev->gem.mutex);
mutex_init(&qdev->update_area_mutex);
mutex_init(&qdev->release_mutex);
iounmap(qdev->ram_header);
iounmap(qdev->rom);
qdev->rom = NULL;
- qdev->mode_info.modes = NULL;
- qdev->mode_info.num_modes = 0;
- qxl_debugfs_remove_files(qdev);
}
return bo;
}
-int qxl_bo_pin(struct qxl_bo *bo, u32 domain, u64 *gpu_addr)
+int __qxl_bo_pin(struct qxl_bo *bo, u32 domain, u64 *gpu_addr)
{
struct drm_device *ddev = bo->gem_base.dev;
int r;
return r;
}
-int qxl_bo_unpin(struct qxl_bo *bo)
+int __qxl_bo_unpin(struct qxl_bo *bo)
{
struct drm_device *ddev = bo->gem_base.dev;
int r, i;
return r;
}
+
+/*
+ * Reserve the BO before pinning the object. If the BO was reserved
+ * beforehand, use the internal version directly __qxl_bo_pin.
+ *
+ */
+int qxl_bo_pin(struct qxl_bo *bo, u32 domain, u64 *gpu_addr)
+{
+ int r;
+
+ r = qxl_bo_reserve(bo, false);
+ if (r)
+ return r;
+
+ r = __qxl_bo_pin(bo, bo->type, NULL);
+ qxl_bo_unreserve(bo);
+ return r;
+}
+
+/*
+ * Reserve the BO before pinning the object. If the BO was reserved
+ * beforehand, use the internal version directly __qxl_bo_unpin.
+ *
+ */
+int qxl_bo_unpin(struct qxl_bo *bo)
+{
+ int r;
+
+ r = qxl_bo_reserve(bo, false);
+ if (r)
+ return r;
+
+ r = __qxl_bo_unpin(bo);
+ qxl_bo_unreserve(bo);
+ return r;
+}
+
void qxl_bo_force_delete(struct qxl_device *qdev)
{
struct qxl_bo *bo, *n;
pdev = platform_device_register_simple("r128_cce", 0, NULL, 0);
if (IS_ERR(pdev)) {
- printk(KERN_ERR "r128_cce: Failed to register firmware\n");
+ pr_err("r128_cce: Failed to register firmware\n");
return PTR_ERR(pdev);
}
rc = request_firmware(&fw, FIRMWARE_NAME, &pdev->dev);
platform_device_unregister(pdev);
if (rc) {
- printk(KERN_ERR "r128_cce: Failed to load firmware \"%s\"\n",
+ pr_err("r128_cce: Failed to load firmware \"%s\"\n",
FIRMWARE_NAME);
return rc;
}
if (fw->size != 256 * 8) {
- printk(KERN_ERR
- "r128_cce: Bogus length %zu in firmware \"%s\"\n",
+ pr_err("r128_cce: Bogus length %zu in firmware \"%s\"\n",
fw->size, FIRMWARE_NAME);
rc = -EINVAL;
goto out_release;
info = drm_fb_helper_alloc_fbi(helper);
if (IS_ERR(info)) {
ret = PTR_ERR(info);
- goto out_unref;
+ goto out;
}
info->par = rfbdev;
ret = radeon_framebuffer_init(rdev->ddev, &rfbdev->rfb, &mode_cmd, gobj);
if (ret) {
DRM_ERROR("failed to initialize framebuffer %d\n", ret);
- goto out_destroy_fbi;
+ goto out;
}
fb = &rfbdev->rfb.base;
if (info->screen_base == NULL) {
ret = -ENOSPC;
- goto out_destroy_fbi;
+ goto out;
}
DRM_INFO("fb mappable at 0x%lX\n", info->fix.smem_start);
vga_switcheroo_client_fb_set(rdev->ddev->pdev, info);
return 0;
-out_destroy_fbi:
- drm_fb_helper_release_fbi(helper);
-out_unref:
+out:
if (rbo) {
}
struct radeon_framebuffer *rfb = &rfbdev->rfb;
drm_fb_helper_unregister_fbi(&rfbdev->helper);
- drm_fb_helper_release_fbi(&rfbdev->helper);
if (rfb->obj) {
radeonfb_destroy_pinned_object(rfb->obj);
.atomic_flush = rcar_du_crtc_atomic_flush,
};
+static int rcar_du_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ struct rcar_du_crtc *rcrtc = to_rcar_crtc(crtc);
+
+ rcar_du_crtc_write(rcrtc, DSRCR, DSRCR_VBCL);
+ rcar_du_crtc_set(rcrtc, DIER, DIER_VBE);
+
+ return 0;
+}
+
+static void rcar_du_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+ struct rcar_du_crtc *rcrtc = to_rcar_crtc(crtc);
+
+ rcar_du_crtc_clr(rcrtc, DIER, DIER_VBE);
+}
+
static const struct drm_crtc_funcs crtc_funcs = {
.reset = drm_atomic_helper_crtc_reset,
.destroy = drm_crtc_cleanup,
.page_flip = drm_atomic_helper_page_flip,
.atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .enable_vblank = rcar_du_crtc_enable_vblank,
+ .disable_vblank = rcar_du_crtc_disable_vblank,
};
/* -----------------------------------------------------------------------------
return 0;
}
-
-void rcar_du_crtc_enable_vblank(struct rcar_du_crtc *rcrtc, bool enable)
-{
- if (enable) {
- rcar_du_crtc_write(rcrtc, DSRCR, DSRCR_VBCL);
- rcar_du_crtc_set(rcrtc, DIER, DIER_VBE);
- } else {
- rcar_du_crtc_clr(rcrtc, DIER, DIER_VBE);
- }
-}
};
int rcar_du_crtc_create(struct rcar_du_group *rgrp, unsigned int index);
-void rcar_du_crtc_enable_vblank(struct rcar_du_crtc *rcrtc, bool enable);
void rcar_du_crtc_suspend(struct rcar_du_crtc *rcrtc);
void rcar_du_crtc_resume(struct rcar_du_crtc *rcrtc);
#include <drm/drm_fb_cma_helper.h>
#include <drm/drm_gem_cma_helper.h>
-#include "rcar_du_crtc.h"
#include "rcar_du_drv.h"
#include "rcar_du_kms.h"
#include "rcar_du_regs.h"
drm_fbdev_cma_restore_mode(rcdu->fbdev);
}
-static int rcar_du_enable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- struct rcar_du_device *rcdu = dev->dev_private;
-
- rcar_du_crtc_enable_vblank(&rcdu->crtcs[pipe], true);
-
- return 0;
-}
-
-static void rcar_du_disable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- struct rcar_du_device *rcdu = dev->dev_private;
-
- rcar_du_crtc_enable_vblank(&rcdu->crtcs[pipe], false);
-}
-
-static const struct file_operations rcar_du_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .poll = drm_poll,
- .read = drm_read,
- .llseek = no_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(rcar_du_fops);
static struct drm_driver rcar_du_driver = {
.driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_PRIME
| DRIVER_ATOMIC,
.lastclose = rcar_du_lastclose,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = rcar_du_enable_vblank,
- .disable_vblank = rcar_du_disable_vblank,
.gem_free_object_unlocked = drm_gem_cma_free_object,
.gem_vm_ops = &drm_gem_cma_vm_ops,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
static int cdn_dp_clk_enable(struct cdn_dp_device *dp)
{
int ret;
- u32 rate;
+ unsigned long rate;
ret = clk_prepare_enable(dp->pclk);
if (ret < 0) {
rate = clk_get_rate(dp->core_clk);
if (!rate) {
- DRM_DEV_ERROR(dp->dev, "get clk rate failed: %d\n", rate);
+ DRM_DEV_ERROR(dp->dev, "get clk rate failed\n");
+ ret = -EINVAL;
goto err_set_rate;
}
#define LINK_TRAINING_RETRY_MS 20
#define LINK_TRAINING_TIMEOUT_MS 500
-void cdn_dp_set_fw_clk(struct cdn_dp_device *dp, u32 clk)
+void cdn_dp_set_fw_clk(struct cdn_dp_device *dp, unsigned long clk)
{
writel(clk / 1000000, dp->regs + SW_CLK_H);
}
rem = do_div(symbol, 1000);
if (tu_size_reg > 64) {
ret = -EINVAL;
+ DRM_DEV_ERROR(dp->dev,
+ "tu error, clk:%d, lanes:%d, rate:%d\n",
+ mode->clock, dp->link.num_lanes,
+ link_rate);
goto err_config_video;
}
} while ((symbol <= 1) || (tu_size_reg - symbol < 4) ||
/* dptx phy addr */
#define DP_TX_PHY_CONFIG_REG 0x2000
-#define DP_TX_PHY_STATUS_REG 0x2004
-#define DP_TX_PHY_SW_RESET 0x2008
-#define DP_TX_PHY_SCRAMBLER_SEED 0x200c
-#define DP_TX_PHY_TRAINING_01_04 0x2010
-#define DP_TX_PHY_TRAINING_05_08 0x2014
-#define DP_TX_PHY_TRAINING_09_10 0x2018
+#define DP_TX_PHY_SW_RESET 0x2004
+#define DP_TX_PHY_SCRAMBLER_SEED 0x2008
+#define DP_TX_PHY_TRAINING_01_04 0x200c
+#define DP_TX_PHY_TRAINING_05_08 0x2010
+#define DP_TX_PHY_TRAINING_09_10 0x2014
#define TEST_COR 0x23fc
/* dptx hpd addr */
void cdn_dp_clock_reset(struct cdn_dp_device *dp);
-void cdn_dp_set_fw_clk(struct cdn_dp_device *dp, u32 clk);
+void cdn_dp_set_fw_clk(struct cdn_dp_device *dp, unsigned long clk);
int cdn_dp_load_firmware(struct cdn_dp_device *dp, const u32 *i_mem,
u32 i_size, const u32 *d_mem, u32 d_size);
int cdn_dp_set_firmware_active(struct cdn_dp_device *dp, bool enable);
#include <linux/math64.h>
#include <linux/module.h>
#include <linux/of_device.h>
+#include <linux/pm_runtime.h>
#include <linux/regmap.h>
+#include <linux/reset.h>
#include <linux/mfd/syscon.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_crtc.h>
#define DRIVER_NAME "dw-mipi-dsi"
-#define GRF_SOC_CON6 0x025c
-#define DSI0_SEL_VOP_LIT (1 << 6)
-#define DSI1_SEL_VOP_LIT (1 << 9)
+#define RK3288_GRF_SOC_CON6 0x025c
+#define RK3288_DSI0_SEL_VOP_LIT BIT(6)
+#define RK3288_DSI1_SEL_VOP_LIT BIT(9)
+
+#define RK3399_GRF_SOC_CON19 0x6250
+#define RK3399_DSI0_SEL_VOP_LIT BIT(0)
+#define RK3399_DSI1_SEL_VOP_LIT BIT(4)
+
+/* disable turnrequest, turndisable, forcetxstopmode, forcerxmode */
+#define RK3399_GRF_SOC_CON22 0x6258
+#define RK3399_GRF_DSI_MODE 0xffff0000
#define DSI_VERSION 0x00
#define DSI_PWR_UP 0x04
#define FRAME_BTA_ACK BIT(14)
#define ENABLE_LOW_POWER (0x3f << 8)
#define ENABLE_LOW_POWER_MASK (0x3f << 8)
-#define VID_MODE_TYPE_BURST_SYNC_PULSES 0x2
+#define VID_MODE_TYPE_NON_BURST_SYNC_PULSES 0x0
+#define VID_MODE_TYPE_NON_BURST_SYNC_EVENTS 0x1
+#define VID_MODE_TYPE_BURST 0x2
#define VID_MODE_TYPE_MASK 0x3
#define DSI_VID_PKT_SIZE 0x3c
#define LPRX_TO_CNT(p) ((p) & 0xffff)
#define DSI_BTA_TO_CNT 0x8c
-
#define DSI_LPCLK_CTRL 0x94
#define AUTO_CLKLANE_CTRL BIT(1)
#define PHY_TXREQUESTCLKHS BIT(0)
#define HSFREQRANGE_SEL(val) (((val) & 0x3f) << 1)
-#define INPUT_DIVIDER(val) ((val - 1) & 0x7f)
+#define INPUT_DIVIDER(val) (((val) - 1) & 0x7f)
#define LOW_PROGRAM_EN 0
#define HIGH_PROGRAM_EN BIT(7)
-#define LOOP_DIV_LOW_SEL(val) ((val - 1) & 0x1f)
-#define LOOP_DIV_HIGH_SEL(val) (((val - 1) >> 5) & 0x1f)
+#define LOOP_DIV_LOW_SEL(val) (((val) - 1) & 0x1f)
+#define LOOP_DIV_HIGH_SEL(val) ((((val) - 1) >> 5) & 0x1f)
#define PLL_LOOP_DIV_EN BIT(5)
#define PLL_INPUT_DIV_EN BIT(4)
};
struct dw_mipi_dsi_plat_data {
+ u32 dsi0_en_bit;
+ u32 dsi1_en_bit;
+ u32 grf_switch_reg;
+ u32 grf_dsi0_mode;
+ u32 grf_dsi0_mode_reg;
unsigned int max_data_lanes;
- enum drm_mode_status (*mode_valid)(struct drm_connector *connector,
- struct drm_display_mode *mode);
};
struct dw_mipi_dsi {
struct clk *pllref_clk;
struct clk *pclk;
+ struct clk *phy_cfg_clk;
+ int dpms_mode;
unsigned int lane_mbps; /* per lane */
u32 channel;
u32 lanes;
u32 format;
u16 input_div;
u16 feedback_div;
- struct drm_display_mode *mode;
+ unsigned long mode_flags;
const struct dw_mipi_dsi_plat_data *pdata;
};
* The controller should generate 2 frames before
* preparing the peripheral.
*/
-static void dw_mipi_dsi_wait_for_two_frames(struct dw_mipi_dsi *dsi)
+static void dw_mipi_dsi_wait_for_two_frames(struct drm_display_mode *mode)
{
int refresh, two_frames;
- refresh = drm_mode_vrefresh(dsi->mode);
+ refresh = drm_mode_vrefresh(mode);
two_frames = DIV_ROUND_UP(MSEC_PER_SEC, refresh) * 2;
msleep(two_frames);
}
{
return container_of(encoder, struct dw_mipi_dsi, encoder);
}
+
static inline void dsi_write(struct dw_mipi_dsi *dsi, u32 reg, u32 val)
{
writel(val, dsi->base + reg);
}
static void dw_mipi_dsi_phy_write(struct dw_mipi_dsi *dsi, u8 test_code,
- u8 test_data)
+ u8 test_data)
{
/*
* With the falling edge on TESTCLK, the TESTDIN[7:0] signal content
dsi_write(dsi, DSI_PHY_TST_CTRL0, PHY_TESTCLK | PHY_UNTESTCLR);
}
+/**
+ * ns2bc - Nanoseconds to byte clock cycles
+ */
+static inline unsigned int ns2bc(struct dw_mipi_dsi *dsi, int ns)
+{
+ return DIV_ROUND_UP(ns * dsi->lane_mbps / 8, 1000);
+}
+
+/**
+ * ns2ui - Nanoseconds to UI time periods
+ */
+static inline unsigned int ns2ui(struct dw_mipi_dsi *dsi, int ns)
+{
+ return DIV_ROUND_UP(ns * dsi->lane_mbps, 1000);
+}
+
static int dw_mipi_dsi_phy_init(struct dw_mipi_dsi *dsi)
{
int ret, testdin, vco, val;
return testdin;
}
- dsi_write(dsi, DSI_PWR_UP, POWERUP);
+ /* Start by clearing PHY state */
+ dsi_write(dsi, DSI_PHY_TST_CTRL0, PHY_UNTESTCLR);
+ dsi_write(dsi, DSI_PHY_TST_CTRL0, PHY_TESTCLR);
+ dsi_write(dsi, DSI_PHY_TST_CTRL0, PHY_UNTESTCLR);
+
+ ret = clk_prepare_enable(dsi->phy_cfg_clk);
+ if (ret) {
+ dev_err(dsi->dev, "Failed to enable phy_cfg_clk\n");
+ return ret;
+ }
dw_mipi_dsi_phy_write(dsi, 0x10, BYPASS_VCO_RANGE |
VCO_RANGE_CON_SEL(vco) |
dw_mipi_dsi_phy_write(dsi, 0x44, HSFREQRANGE_SEL(testdin));
- dw_mipi_dsi_phy_write(dsi, 0x19, PLL_LOOP_DIV_EN | PLL_INPUT_DIV_EN);
dw_mipi_dsi_phy_write(dsi, 0x17, INPUT_DIVIDER(dsi->input_div));
dw_mipi_dsi_phy_write(dsi, 0x18, LOOP_DIV_LOW_SEL(dsi->feedback_div) |
LOW_PROGRAM_EN);
dw_mipi_dsi_phy_write(dsi, 0x18, LOOP_DIV_HIGH_SEL(dsi->feedback_div) |
HIGH_PROGRAM_EN);
+ dw_mipi_dsi_phy_write(dsi, 0x19, PLL_LOOP_DIV_EN | PLL_INPUT_DIV_EN);
+
+ dw_mipi_dsi_phy_write(dsi, 0x22, LOW_PROGRAM_EN |
+ BIASEXTR_SEL(BIASEXTR_127_7));
+ dw_mipi_dsi_phy_write(dsi, 0x22, HIGH_PROGRAM_EN |
+ BANDGAP_SEL(BANDGAP_96_10));
dw_mipi_dsi_phy_write(dsi, 0x20, POWER_CONTROL | INTERNAL_REG_CURRENT |
BIAS_BLOCK_ON | BANDGAP_ON);
SETRD_MAX | POWER_MANAGE |
TER_RESISTORS_ON);
- dw_mipi_dsi_phy_write(dsi, 0x22, LOW_PROGRAM_EN |
- BIASEXTR_SEL(BIASEXTR_127_7));
- dw_mipi_dsi_phy_write(dsi, 0x22, HIGH_PROGRAM_EN |
- BANDGAP_SEL(BANDGAP_96_10));
-
- dw_mipi_dsi_phy_write(dsi, 0x70, TLP_PROGRAM_EN | 0xf);
- dw_mipi_dsi_phy_write(dsi, 0x71, THS_PRE_PROGRAM_EN | 0x55);
- dw_mipi_dsi_phy_write(dsi, 0x72, THS_ZERO_PROGRAM_EN | 0xa);
+ dw_mipi_dsi_phy_write(dsi, 0x60, TLP_PROGRAM_EN | ns2bc(dsi, 500));
+ dw_mipi_dsi_phy_write(dsi, 0x61, THS_PRE_PROGRAM_EN | ns2ui(dsi, 40));
+ dw_mipi_dsi_phy_write(dsi, 0x62, THS_ZERO_PROGRAM_EN | ns2bc(dsi, 300));
+ dw_mipi_dsi_phy_write(dsi, 0x63, THS_PRE_PROGRAM_EN | ns2ui(dsi, 100));
+ dw_mipi_dsi_phy_write(dsi, 0x64, BIT(5) | ns2bc(dsi, 100));
+ dw_mipi_dsi_phy_write(dsi, 0x65, BIT(5) | (ns2bc(dsi, 60) + 7));
+
+ dw_mipi_dsi_phy_write(dsi, 0x70, TLP_PROGRAM_EN | ns2bc(dsi, 500));
+ dw_mipi_dsi_phy_write(dsi, 0x71,
+ THS_PRE_PROGRAM_EN | (ns2ui(dsi, 50) + 5));
+ dw_mipi_dsi_phy_write(dsi, 0x72,
+ THS_ZERO_PROGRAM_EN | (ns2bc(dsi, 140) + 2));
+ dw_mipi_dsi_phy_write(dsi, 0x73,
+ THS_PRE_PROGRAM_EN | (ns2ui(dsi, 60) + 8));
+ dw_mipi_dsi_phy_write(dsi, 0x74, BIT(5) | ns2bc(dsi, 100));
dsi_write(dsi, DSI_PHY_RSTZ, PHY_ENFORCEPLL | PHY_ENABLECLK |
PHY_UNRSTZ | PHY_UNSHUTDOWNZ);
-
- ret = readx_poll_timeout(readl, dsi->base + DSI_PHY_STATUS,
+ ret = readl_poll_timeout(dsi->base + DSI_PHY_STATUS,
val, val & LOCK, 1000, PHY_STATUS_TIMEOUT_US);
if (ret < 0) {
dev_err(dsi->dev, "failed to wait for phy lock state\n");
- return ret;
+ goto phy_init_end;
}
- ret = readx_poll_timeout(readl, dsi->base + DSI_PHY_STATUS,
+ ret = readl_poll_timeout(dsi->base + DSI_PHY_STATUS,
val, val & STOP_STATE_CLK_LANE, 1000,
PHY_STATUS_TIMEOUT_US);
- if (ret < 0) {
+ if (ret < 0)
dev_err(dsi->dev,
"failed to wait for phy clk lane stop state\n");
- return ret;
- }
+
+phy_init_end:
+ clk_disable_unprepare(dsi->phy_cfg_clk);
return ret;
}
-static int dw_mipi_dsi_get_lane_bps(struct dw_mipi_dsi *dsi)
+static int dw_mipi_dsi_get_lane_bps(struct dw_mipi_dsi *dsi,
+ struct drm_display_mode *mode)
{
unsigned int i, pre;
unsigned long mpclk, pllref, tmp;
return bpp;
}
- mpclk = DIV_ROUND_UP(dsi->mode->clock, MSEC_PER_SEC);
+ mpclk = DIV_ROUND_UP(mode->clock, MSEC_PER_SEC);
if (mpclk) {
- /* take 1 / 0.9, since mbps must big than bandwidth of RGB */
- tmp = mpclk * (bpp / dsi->lanes) * 10 / 9;
+ /* take 1 / 0.8, since mbps must big than bandwidth of RGB */
+ tmp = mpclk * (bpp / dsi->lanes) * 10 / 8;
if (tmp < max_mbps)
target_mbps = tmp;
else
pllref = DIV_ROUND_UP(clk_get_rate(dsi->pllref_clk), USEC_PER_SEC);
tmp = pllref;
- for (i = 1; i < 6; i++) {
+ /*
+ * The limits on the PLL divisor are:
+ *
+ * 5MHz <= (pllref / n) <= 40MHz
+ *
+ * we walk over these values in descreasing order so that if we hit
+ * an exact match for target_mbps it is more likely that "m" will be
+ * even.
+ *
+ * TODO: ensure that "m" is even after this loop.
+ */
+ for (i = pllref / 5; i > (pllref / 40); i--) {
pre = pllref / i;
if ((tmp > (target_mbps % pre)) && (target_mbps / pre < 512)) {
tmp = target_mbps % pre;
if (device->lanes > dsi->pdata->max_data_lanes) {
dev_err(dsi->dev, "the number of data lanes(%u) is too many\n",
- device->lanes);
- return -EINVAL;
- }
-
- if (!(device->mode_flags & MIPI_DSI_MODE_VIDEO_BURST) ||
- !(device->mode_flags & MIPI_DSI_MODE_VIDEO_SYNC_PULSE)) {
- dev_err(dsi->dev, "device mode is unsupported\n");
+ device->lanes);
return -EINVAL;
}
dsi->lanes = device->lanes;
dsi->channel = device->channel;
dsi->format = device->format;
+ dsi->mode_flags = device->mode_flags;
dsi->panel = of_drm_find_panel(device->dev.of_node);
if (dsi->panel)
return drm_panel_attach(dsi->panel, &dsi->connector);
return 0;
}
-static int dw_mipi_dsi_gen_pkt_hdr_write(struct dw_mipi_dsi *dsi, u32 val)
+static void dw_mipi_message_config(struct dw_mipi_dsi *dsi,
+ const struct mipi_dsi_msg *msg)
+{
+ bool lpm = msg->flags & MIPI_DSI_MSG_USE_LPM;
+ u32 val = 0;
+
+ if (msg->flags & MIPI_DSI_MSG_REQ_ACK)
+ val |= EN_ACK_RQST;
+ if (lpm)
+ val |= CMD_MODE_ALL_LP;
+
+ dsi_write(dsi, DSI_LPCLK_CTRL, lpm ? 0 : PHY_TXREQUESTCLKHS);
+ dsi_write(dsi, DSI_CMD_MODE_CFG, val);
+}
+
+static int dw_mipi_dsi_gen_pkt_hdr_write(struct dw_mipi_dsi *dsi, u32 hdr_val)
{
int ret;
+ u32 val, mask;
- ret = readx_poll_timeout(readl, dsi->base + DSI_CMD_PKT_STATUS,
+ ret = readl_poll_timeout(dsi->base + DSI_CMD_PKT_STATUS,
val, !(val & GEN_CMD_FULL), 1000,
CMD_PKT_STATUS_TIMEOUT_US);
if (ret < 0) {
return ret;
}
- dsi_write(dsi, DSI_GEN_HDR, val);
+ dsi_write(dsi, DSI_GEN_HDR, hdr_val);
- ret = readx_poll_timeout(readl, dsi->base + DSI_CMD_PKT_STATUS,
- val, val & (GEN_CMD_EMPTY | GEN_PLD_W_EMPTY),
+ mask = GEN_CMD_EMPTY | GEN_PLD_W_EMPTY;
+ ret = readl_poll_timeout(dsi->base + DSI_CMD_PKT_STATUS,
+ val, (val & mask) == mask,
1000, CMD_PKT_STATUS_TIMEOUT_US);
if (ret < 0) {
dev_err(dsi->dev, "failed to write command FIFO\n");
static int dw_mipi_dsi_dcs_short_write(struct dw_mipi_dsi *dsi,
const struct mipi_dsi_msg *msg)
{
- const u16 *tx_buf = msg->tx_buf;
- u32 val = GEN_HDATA(*tx_buf) | GEN_HTYPE(msg->type);
+ const u8 *tx_buf = msg->tx_buf;
+ u16 data = 0;
+ u32 val;
+
+ if (msg->tx_len > 0)
+ data |= tx_buf[0];
+ if (msg->tx_len > 1)
+ data |= tx_buf[1] << 8;
if (msg->tx_len > 2) {
dev_err(dsi->dev, "too long tx buf length %zu for short write\n",
return -EINVAL;
}
+ val = GEN_HDATA(data) | GEN_HTYPE(msg->type);
return dw_mipi_dsi_gen_pkt_hdr_write(dsi, val);
}
static int dw_mipi_dsi_dcs_long_write(struct dw_mipi_dsi *dsi,
const struct mipi_dsi_msg *msg)
{
- const u32 *tx_buf = msg->tx_buf;
- int len = msg->tx_len, pld_data_bytes = sizeof(*tx_buf), ret;
- u32 val = GEN_HDATA(msg->tx_len) | GEN_HTYPE(msg->type);
- u32 remainder = 0;
+ const u8 *tx_buf = msg->tx_buf;
+ int len = msg->tx_len, pld_data_bytes = sizeof(u32), ret;
+ u32 hdr_val = GEN_HDATA(msg->tx_len) | GEN_HTYPE(msg->type);
+ u32 remainder;
+ u32 val;
if (msg->tx_len < 3) {
dev_err(dsi->dev, "wrong tx buf length %zu for long write\n",
while (DIV_ROUND_UP(len, pld_data_bytes)) {
if (len < pld_data_bytes) {
+ remainder = 0;
memcpy(&remainder, tx_buf, len);
dsi_write(dsi, DSI_GEN_PLD_DATA, remainder);
len = 0;
} else {
- dsi_write(dsi, DSI_GEN_PLD_DATA, *tx_buf);
- tx_buf++;
+ memcpy(&remainder, tx_buf, pld_data_bytes);
+ dsi_write(dsi, DSI_GEN_PLD_DATA, remainder);
+ tx_buf += pld_data_bytes;
len -= pld_data_bytes;
}
- ret = readx_poll_timeout(readl, dsi->base + DSI_CMD_PKT_STATUS,
+ ret = readl_poll_timeout(dsi->base + DSI_CMD_PKT_STATUS,
val, !(val & GEN_PLD_W_FULL), 1000,
CMD_PKT_STATUS_TIMEOUT_US);
if (ret < 0) {
}
}
- return dw_mipi_dsi_gen_pkt_hdr_write(dsi, val);
+ return dw_mipi_dsi_gen_pkt_hdr_write(dsi, hdr_val);
}
static ssize_t dw_mipi_dsi_host_transfer(struct mipi_dsi_host *host,
struct dw_mipi_dsi *dsi = host_to_dsi(host);
int ret;
+ dw_mipi_message_config(dsi, msg);
+
switch (msg->type) {
case MIPI_DSI_DCS_SHORT_WRITE:
case MIPI_DSI_DCS_SHORT_WRITE_PARAM:
ret = dw_mipi_dsi_dcs_long_write(dsi, msg);
break;
default:
- dev_err(dsi->dev, "unsupported message type\n");
+ dev_err(dsi->dev, "unsupported message type 0x%02x\n",
+ msg->type);
ret = -EINVAL;
}
{
u32 val;
- val = VID_MODE_TYPE_BURST_SYNC_PULSES | ENABLE_LOW_POWER;
+ val = ENABLE_LOW_POWER;
+
+ if (dsi->mode_flags & MIPI_DSI_MODE_VIDEO_BURST)
+ val |= VID_MODE_TYPE_BURST;
+ else if (dsi->mode_flags & MIPI_DSI_MODE_VIDEO_SYNC_PULSE)
+ val |= VID_MODE_TYPE_NON_BURST_SYNC_PULSES;
+ else
+ val |= VID_MODE_TYPE_NON_BURST_SYNC_EVENTS;
dsi_write(dsi, DSI_VID_MODE_CFG, val);
}
dsi_write(dsi, DSI_PWR_UP, RESET);
dsi_write(dsi, DSI_MODE_CFG, ENABLE_VIDEO_MODE);
dw_mipi_dsi_video_mode_config(dsi);
+ dsi_write(dsi, DSI_LPCLK_CTRL, PHY_TXREQUESTCLKHS);
dsi_write(dsi, DSI_PWR_UP, POWERUP);
}
}
static void dw_mipi_dsi_init(struct dw_mipi_dsi *dsi)
{
+ /*
+ * The maximum permitted escape clock is 20MHz and it is derived from
+ * lanebyteclk, which is running at "lane_mbps / 8". Thus we want:
+ *
+ * (lane_mbps >> 3) / esc_clk_division < 20
+ * which is:
+ * (lane_mbps >> 3) / 20 > esc_clk_division
+ */
+ u32 esc_clk_division = (dsi->lane_mbps >> 3) / 20 + 1;
+
dsi_write(dsi, DSI_PWR_UP, RESET);
dsi_write(dsi, DSI_PHY_RSTZ, PHY_DISFORCEPLL | PHY_DISABLECLK
| PHY_RSTZ | PHY_SHUTDOWNZ);
dsi_write(dsi, DSI_CLKMGR_CFG, TO_CLK_DIVIDSION(10) |
- TX_ESC_CLK_DIVIDSION(7));
- dsi_write(dsi, DSI_LPCLK_CTRL, PHY_TXREQUESTCLKHS);
+ TX_ESC_CLK_DIVIDSION(esc_clk_division));
}
static void dw_mipi_dsi_dpi_config(struct dw_mipi_dsi *dsi,
break;
}
- if (!(mode->flags & DRM_MODE_FLAG_PVSYNC))
+ if (mode->flags & DRM_MODE_FLAG_NVSYNC)
val |= VSYNC_ACTIVE_LOW;
- if (!(mode->flags & DRM_MODE_FLAG_PHSYNC))
+ if (mode->flags & DRM_MODE_FLAG_NHSYNC)
val |= HSYNC_ACTIVE_LOW;
dsi_write(dsi, DSI_DPI_VCID, DPI_VID(dsi->channel));
{
dsi_write(dsi, DSI_TO_CNT_CFG, HSTX_TO_CNT(1000) | LPRX_TO_CNT(1000));
dsi_write(dsi, DSI_BTA_TO_CNT, 0xd00);
- dsi_write(dsi, DSI_CMD_MODE_CFG, CMD_MODE_ALL_LP);
dsi_write(dsi, DSI_MODE_CFG, ENABLE_CMD_MODE);
}
/* Get lane byte clock cycles. */
static u32 dw_mipi_dsi_get_hcomponent_lbcc(struct dw_mipi_dsi *dsi,
+ struct drm_display_mode *mode,
u32 hcomponent)
{
u32 frac, lbcc;
lbcc = hcomponent * dsi->lane_mbps * MSEC_PER_SEC / 8;
- frac = lbcc % dsi->mode->clock;
- lbcc = lbcc / dsi->mode->clock;
+ frac = lbcc % mode->clock;
+ lbcc = lbcc / mode->clock;
if (frac)
lbcc++;
return lbcc;
}
-static void dw_mipi_dsi_line_timer_config(struct dw_mipi_dsi *dsi)
+static void dw_mipi_dsi_line_timer_config(struct dw_mipi_dsi *dsi,
+ struct drm_display_mode *mode)
{
u32 htotal, hsa, hbp, lbcc;
- struct drm_display_mode *mode = dsi->mode;
htotal = mode->htotal;
hsa = mode->hsync_end - mode->hsync_start;
hbp = mode->htotal - mode->hsync_end;
- lbcc = dw_mipi_dsi_get_hcomponent_lbcc(dsi, htotal);
+ lbcc = dw_mipi_dsi_get_hcomponent_lbcc(dsi, mode, htotal);
dsi_write(dsi, DSI_VID_HLINE_TIME, lbcc);
- lbcc = dw_mipi_dsi_get_hcomponent_lbcc(dsi, hsa);
+ lbcc = dw_mipi_dsi_get_hcomponent_lbcc(dsi, mode, hsa);
dsi_write(dsi, DSI_VID_HSA_TIME, lbcc);
- lbcc = dw_mipi_dsi_get_hcomponent_lbcc(dsi, hbp);
+ lbcc = dw_mipi_dsi_get_hcomponent_lbcc(dsi, mode, hbp);
dsi_write(dsi, DSI_VID_HBP_TIME, lbcc);
}
-static void dw_mipi_dsi_vertical_timing_config(struct dw_mipi_dsi *dsi)
+static void dw_mipi_dsi_vertical_timing_config(struct dw_mipi_dsi *dsi,
+ struct drm_display_mode *mode)
{
u32 vactive, vsa, vfp, vbp;
- struct drm_display_mode *mode = dsi->mode;
vactive = mode->vdisplay;
vsa = mode->vsync_end - mode->vsync_start;
dsi_write(dsi, DSI_INT_MSK1, 0);
}
-static void dw_mipi_dsi_encoder_mode_set(struct drm_encoder *encoder,
- struct drm_display_mode *mode,
- struct drm_display_mode *adjusted_mode)
+static void dw_mipi_dsi_encoder_disable(struct drm_encoder *encoder)
{
struct dw_mipi_dsi *dsi = encoder_to_dsi(encoder);
- int ret;
-
- dsi->mode = adjusted_mode;
- ret = dw_mipi_dsi_get_lane_bps(dsi);
- if (ret < 0)
+ if (dsi->dpms_mode != DRM_MODE_DPMS_ON)
return;
if (clk_prepare_enable(dsi->pclk)) {
return;
}
- dw_mipi_dsi_init(dsi);
- dw_mipi_dsi_dpi_config(dsi, mode);
- dw_mipi_dsi_packet_handler_config(dsi);
- dw_mipi_dsi_video_mode_config(dsi);
- dw_mipi_dsi_video_packet_config(dsi, mode);
- dw_mipi_dsi_command_mode_config(dsi);
- dw_mipi_dsi_line_timer_config(dsi);
- dw_mipi_dsi_vertical_timing_config(dsi);
- dw_mipi_dsi_dphy_timing_config(dsi);
- dw_mipi_dsi_dphy_interface_config(dsi);
- dw_mipi_dsi_clear_err(dsi);
- if (drm_panel_prepare(dsi->panel))
- dev_err(dsi->dev, "failed to prepare panel\n");
-
- clk_disable_unprepare(dsi->pclk);
-}
-
-static void dw_mipi_dsi_encoder_disable(struct drm_encoder *encoder)
-{
- struct dw_mipi_dsi *dsi = encoder_to_dsi(encoder);
-
drm_panel_disable(dsi->panel);
- if (clk_prepare_enable(dsi->pclk)) {
- dev_err(dsi->dev, "%s: Failed to enable pclk\n", __func__);
- return;
- }
-
dw_mipi_dsi_set_mode(dsi, DW_MIPI_DSI_CMD_MODE);
drm_panel_unprepare(dsi->panel);
- dw_mipi_dsi_set_mode(dsi, DW_MIPI_DSI_VID_MODE);
-
- /*
- * This is necessary to make sure the peripheral will be driven
- * normally when the display is enabled again later.
- */
- msleep(120);
- dw_mipi_dsi_set_mode(dsi, DW_MIPI_DSI_CMD_MODE);
dw_mipi_dsi_disable(dsi);
+ pm_runtime_put(dsi->dev);
clk_disable_unprepare(dsi->pclk);
+ dsi->dpms_mode = DRM_MODE_DPMS_OFF;
}
-static void dw_mipi_dsi_encoder_commit(struct drm_encoder *encoder)
+static void dw_mipi_dsi_encoder_enable(struct drm_encoder *encoder)
{
struct dw_mipi_dsi *dsi = encoder_to_dsi(encoder);
+ struct drm_display_mode *mode = &encoder->crtc->state->adjusted_mode;
+ const struct dw_mipi_dsi_plat_data *pdata = dsi->pdata;
int mux = drm_of_encoder_active_endpoint_id(dsi->dev->of_node, encoder);
u32 val;
+ int ret;
+
+ ret = dw_mipi_dsi_get_lane_bps(dsi, mode);
+ if (ret < 0)
+ return;
+
+ if (dsi->dpms_mode == DRM_MODE_DPMS_ON)
+ return;
if (clk_prepare_enable(dsi->pclk)) {
dev_err(dsi->dev, "%s: Failed to enable pclk\n", __func__);
return;
}
+ pm_runtime_get_sync(dsi->dev);
+ dw_mipi_dsi_init(dsi);
+ dw_mipi_dsi_dpi_config(dsi, mode);
+ dw_mipi_dsi_packet_handler_config(dsi);
+ dw_mipi_dsi_video_mode_config(dsi);
+ dw_mipi_dsi_video_packet_config(dsi, mode);
+ dw_mipi_dsi_command_mode_config(dsi);
+ dw_mipi_dsi_line_timer_config(dsi, mode);
+ dw_mipi_dsi_vertical_timing_config(dsi, mode);
+ dw_mipi_dsi_dphy_timing_config(dsi);
+ dw_mipi_dsi_dphy_interface_config(dsi);
+ dw_mipi_dsi_clear_err(dsi);
+
+ if (pdata->grf_dsi0_mode_reg)
+ regmap_write(dsi->grf_regmap, pdata->grf_dsi0_mode_reg,
+ pdata->grf_dsi0_mode);
+
dw_mipi_dsi_phy_init(dsi);
- dw_mipi_dsi_wait_for_two_frames(dsi);
+ dw_mipi_dsi_wait_for_two_frames(mode);
+
+ dw_mipi_dsi_set_mode(dsi, DW_MIPI_DSI_CMD_MODE);
+ if (drm_panel_prepare(dsi->panel))
+ dev_err(dsi->dev, "failed to prepare panel\n");
dw_mipi_dsi_set_mode(dsi, DW_MIPI_DSI_VID_MODE);
drm_panel_enable(dsi->panel);
clk_disable_unprepare(dsi->pclk);
if (mux)
- val = DSI0_SEL_VOP_LIT | (DSI0_SEL_VOP_LIT << 16);
+ val = pdata->dsi0_en_bit | (pdata->dsi0_en_bit << 16);
else
- val = DSI0_SEL_VOP_LIT << 16;
+ val = pdata->dsi0_en_bit << 16;
- regmap_write(dsi->grf_regmap, GRF_SOC_CON6, val);
+ regmap_write(dsi->grf_regmap, pdata->grf_switch_reg, val);
dev_dbg(dsi->dev, "vop %s output to dsi0\n", (mux) ? "LIT" : "BIG");
+ dsi->dpms_mode = DRM_MODE_DPMS_ON;
}
static int
return 0;
}
-static struct drm_encoder_helper_funcs
+static const struct drm_encoder_helper_funcs
dw_mipi_dsi_encoder_helper_funcs = {
- .commit = dw_mipi_dsi_encoder_commit,
- .mode_set = dw_mipi_dsi_encoder_mode_set,
+ .enable = dw_mipi_dsi_encoder_enable,
.disable = dw_mipi_dsi_encoder_disable,
.atomic_check = dw_mipi_dsi_encoder_atomic_check,
};
-static struct drm_encoder_funcs dw_mipi_dsi_encoder_funcs = {
+static const struct drm_encoder_funcs dw_mipi_dsi_encoder_funcs = {
.destroy = drm_encoder_cleanup,
};
return drm_panel_get_modes(dsi->panel);
}
-static enum drm_mode_status dw_mipi_dsi_mode_valid(
- struct drm_connector *connector,
- struct drm_display_mode *mode)
-{
- struct dw_mipi_dsi *dsi = con_to_dsi(connector);
-
- enum drm_mode_status mode_status = MODE_OK;
-
- if (dsi->pdata->mode_valid)
- mode_status = dsi->pdata->mode_valid(connector, mode);
-
- return mode_status;
-}
-
static struct drm_connector_helper_funcs dw_mipi_dsi_connector_helper_funcs = {
.get_modes = dw_mipi_dsi_connector_get_modes,
- .mode_valid = dw_mipi_dsi_mode_valid,
};
static void dw_mipi_dsi_drm_connector_destroy(struct drm_connector *connector)
drm_connector_cleanup(connector);
}
-static struct drm_connector_funcs dw_mipi_dsi_atomic_connector_funcs = {
+static const struct drm_connector_funcs dw_mipi_dsi_atomic_connector_funcs = {
.dpms = drm_atomic_helper_connector_dpms,
.fill_modes = drm_helper_probe_single_connector_modes,
.destroy = dw_mipi_dsi_drm_connector_destroy,
};
static int dw_mipi_dsi_register(struct drm_device *drm,
- struct dw_mipi_dsi *dsi)
+ struct dw_mipi_dsi *dsi)
{
struct drm_encoder *encoder = &dsi->encoder;
struct drm_connector *connector = &dsi->connector;
drm_encoder_helper_add(&dsi->encoder,
&dw_mipi_dsi_encoder_helper_funcs);
ret = drm_encoder_init(drm, &dsi->encoder, &dw_mipi_dsi_encoder_funcs,
- DRM_MODE_ENCODER_DSI, NULL);
+ DRM_MODE_ENCODER_DSI, NULL);
if (ret) {
dev_err(dev, "Failed to initialize encoder with drm\n");
return ret;
}
drm_connector_helper_add(connector,
- &dw_mipi_dsi_connector_helper_funcs);
+ &dw_mipi_dsi_connector_helper_funcs);
drm_connector_init(drm, &dsi->connector,
&dw_mipi_dsi_atomic_connector_funcs,
return 0;
}
-static enum drm_mode_status rk3288_mipi_dsi_mode_valid(
- struct drm_connector *connector,
- struct drm_display_mode *mode)
-{
- /*
- * The VID_PKT_SIZE field in the DSI_VID_PKT_CFG
- * register is 11-bit.
- */
- if (mode->hdisplay > 0x7ff)
- return MODE_BAD_HVALUE;
-
- /*
- * The V_ACTIVE_LINES field in the DSI_VTIMING_CFG
- * register is 11-bit.
- */
- if (mode->vdisplay > 0x7ff)
- return MODE_BAD_VVALUE;
-
- return MODE_OK;
-}
-
static struct dw_mipi_dsi_plat_data rk3288_mipi_dsi_drv_data = {
+ .dsi0_en_bit = RK3288_DSI0_SEL_VOP_LIT,
+ .dsi1_en_bit = RK3288_DSI1_SEL_VOP_LIT,
+ .grf_switch_reg = RK3288_GRF_SOC_CON6,
+ .max_data_lanes = 4,
+};
+
+static struct dw_mipi_dsi_plat_data rk3399_mipi_dsi_drv_data = {
+ .dsi0_en_bit = RK3399_DSI0_SEL_VOP_LIT,
+ .dsi1_en_bit = RK3399_DSI1_SEL_VOP_LIT,
+ .grf_switch_reg = RK3399_GRF_SOC_CON19,
+ .grf_dsi0_mode = RK3399_GRF_DSI_MODE,
+ .grf_dsi0_mode_reg = RK3399_GRF_SOC_CON22,
.max_data_lanes = 4,
- .mode_valid = rk3288_mipi_dsi_mode_valid,
};
static const struct of_device_id dw_mipi_dsi_dt_ids[] = {
{
.compatible = "rockchip,rk3288-mipi-dsi",
.data = &rk3288_mipi_dsi_drv_data,
+ }, {
+ .compatible = "rockchip,rk3399-mipi-dsi",
+ .data = &rk3399_mipi_dsi_drv_data,
},
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(of, dw_mipi_dsi_dt_ids);
static int dw_mipi_dsi_bind(struct device *dev, struct device *master,
- void *data)
+ void *data)
{
const struct of_device_id *of_id =
of_match_device(dw_mipi_dsi_dt_ids, dev);
const struct dw_mipi_dsi_plat_data *pdata = of_id->data;
struct platform_device *pdev = to_platform_device(dev);
+ struct reset_control *apb_rst;
struct drm_device *drm = data;
struct dw_mipi_dsi *dsi;
struct resource *res;
dsi->dev = dev;
dsi->pdata = pdata;
+ dsi->dpms_mode = DRM_MODE_DPMS_OFF;
ret = rockchip_mipi_parse_dt(dsi);
if (ret)
return ret;
}
+ /*
+ * Note that the reset was not defined in the initial device tree, so
+ * we have to be prepared for it not being found.
+ */
+ apb_rst = devm_reset_control_get(dev, "apb");
+ if (IS_ERR(apb_rst)) {
+ ret = PTR_ERR(apb_rst);
+ if (ret == -ENOENT) {
+ apb_rst = NULL;
+ } else {
+ dev_err(dev, "Unable to get reset control: %d\n", ret);
+ return ret;
+ }
+ }
+
+ if (apb_rst) {
+ ret = clk_prepare_enable(dsi->pclk);
+ if (ret) {
+ dev_err(dev, "%s: Failed to enable pclk\n", __func__);
+ return ret;
+ }
+
+ reset_control_assert(apb_rst);
+ usleep_range(10, 20);
+ reset_control_deassert(apb_rst);
+
+ clk_disable_unprepare(dsi->pclk);
+ }
+
+ dsi->phy_cfg_clk = devm_clk_get(dev, "phy_cfg");
+ if (IS_ERR(dsi->phy_cfg_clk)) {
+ ret = PTR_ERR(dsi->phy_cfg_clk);
+ if (ret != -ENOENT) {
+ dev_err(dev, "Unable to get phy_cfg_clk: %d\n", ret);
+ return ret;
+ }
+ dsi->phy_cfg_clk = NULL;
+ dev_dbg(dev, "have not phy_cfg_clk\n");
+ }
+
ret = clk_prepare_enable(dsi->pllref_clk);
if (ret) {
dev_err(dev, "%s: Failed to enable pllref_clk\n", __func__);
goto err_pllref;
}
- dev_set_drvdata(dev, dsi);
+ pm_runtime_enable(dev);
dsi->dsi_host.ops = &dw_mipi_dsi_host_ops;
dsi->dsi_host.dev = dev;
- return mipi_dsi_host_register(&dsi->dsi_host);
+ ret = mipi_dsi_host_register(&dsi->dsi_host);
+ if (ret) {
+ dev_err(dev, "Failed to register MIPI host: %d\n", ret);
+ goto err_cleanup;
+ }
+
+ if (!dsi->panel) {
+ ret = -EPROBE_DEFER;
+ goto err_mipi_dsi_host;
+ }
+ dev_set_drvdata(dev, dsi);
+ return 0;
+
+err_mipi_dsi_host:
+ mipi_dsi_host_unregister(&dsi->dsi_host);
+err_cleanup:
+ drm_encoder_cleanup(&dsi->encoder);
+ drm_connector_cleanup(&dsi->connector);
err_pllref:
clk_disable_unprepare(dsi->pllref_clk);
return ret;
}
static void dw_mipi_dsi_unbind(struct device *dev, struct device *master,
- void *data)
+ void *data)
{
struct dw_mipi_dsi *dsi = dev_get_drvdata(dev);
mipi_dsi_host_unregister(&dsi->dsi_host);
+ pm_runtime_disable(dev);
clk_disable_unprepare(dsi->pllref_clk);
}
.mpll_cfg = rockchip_mpll_cfg,
.cur_ctr = rockchip_cur_ctr,
.phy_config = rockchip_phy_config,
- .dev_type = RK3288_HDMI,
};
static const struct of_device_id dw_hdmi_rockchip_dt_ids[] = {
iommu_detach_device(domain, dev);
}
-int rockchip_register_crtc_funcs(struct drm_crtc *crtc,
- const struct rockchip_crtc_funcs *crtc_funcs)
-{
- int pipe = drm_crtc_index(crtc);
- struct rockchip_drm_private *priv = crtc->dev->dev_private;
-
- if (pipe >= ROCKCHIP_MAX_CRTC)
- return -EINVAL;
-
- priv->crtc_funcs[pipe] = crtc_funcs;
-
- return 0;
-}
-
-void rockchip_unregister_crtc_funcs(struct drm_crtc *crtc)
-{
- int pipe = drm_crtc_index(crtc);
- struct rockchip_drm_private *priv = crtc->dev->dev_private;
-
- if (pipe >= ROCKCHIP_MAX_CRTC)
- return;
-
- priv->crtc_funcs[pipe] = NULL;
-}
-
-static int rockchip_drm_crtc_enable_vblank(struct drm_device *dev,
- unsigned int pipe)
-{
- struct rockchip_drm_private *priv = dev->dev_private;
- struct drm_crtc *crtc = drm_crtc_from_index(dev, pipe);
-
- if (crtc && priv->crtc_funcs[pipe] &&
- priv->crtc_funcs[pipe]->enable_vblank)
- return priv->crtc_funcs[pipe]->enable_vblank(crtc);
-
- return 0;
-}
-
-static void rockchip_drm_crtc_disable_vblank(struct drm_device *dev,
- unsigned int pipe)
-{
- struct rockchip_drm_private *priv = dev->dev_private;
- struct drm_crtc *crtc = drm_crtc_from_index(dev, pipe);
-
- if (crtc && priv->crtc_funcs[pipe] &&
- priv->crtc_funcs[pipe]->enable_vblank)
- priv->crtc_funcs[pipe]->disable_vblank(crtc);
-}
-
static int rockchip_drm_init_iommu(struct drm_device *drm_dev)
{
struct rockchip_drm_private *private = drm_dev->dev_private;
.driver_features = DRIVER_MODESET | DRIVER_GEM |
DRIVER_PRIME | DRIVER_ATOMIC,
.lastclose = rockchip_drm_lastclose,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = rockchip_drm_crtc_enable_vblank,
- .disable_vblank = rockchip_drm_crtc_disable_vblank,
.gem_vm_ops = &drm_gem_cma_vm_ops,
.gem_free_object_unlocked = rockchip_gem_free_object,
.dumb_create = rockchip_gem_dumb_create,
struct drm_connector;
struct iommu_domain;
-/*
- * Rockchip drm private crtc funcs.
- * @enable_vblank: enable crtc vblank irq.
- * @disable_vblank: disable crtc vblank irq.
- */
-struct rockchip_crtc_funcs {
- int (*enable_vblank)(struct drm_crtc *crtc);
- void (*disable_vblank)(struct drm_crtc *crtc);
-};
-
struct rockchip_crtc_state {
struct drm_crtc_state base;
int output_type;
struct rockchip_drm_private {
struct drm_fb_helper fbdev_helper;
struct drm_gem_object *fbdev_bo;
- const struct rockchip_crtc_funcs *crtc_funcs[ROCKCHIP_MAX_CRTC];
struct drm_atomic_state *state;
struct iommu_domain *domain;
/* protect drm_mm on multi-threads */
spinlock_t psr_list_lock;
};
-int rockchip_register_crtc_funcs(struct drm_crtc *crtc,
- const struct rockchip_crtc_funcs *crtc_funcs);
-void rockchip_unregister_crtc_funcs(struct drm_crtc *crtc);
int rockchip_drm_dma_attach_device(struct drm_device *drm_dev,
struct device *dev);
void rockchip_drm_dma_detach_device(struct drm_device *drm_dev,
drm_atomic_helper_cleanup_planes(dev, state);
}
-static struct drm_mode_config_helper_funcs rockchip_mode_config_helpers = {
+static const struct drm_mode_config_helper_funcs rockchip_mode_config_helpers = {
.atomic_commit_tail = rockchip_atomic_commit_tail,
};
if (IS_ERR(fbi)) {
dev_err(dev->dev, "Failed to create framebuffer info.\n");
ret = PTR_ERR(fbi);
- goto err_rockchip_gem_free_object;
+ goto out;
}
helper->fb = rockchip_drm_framebuffer_init(dev, &mode_cmd,
if (IS_ERR(helper->fb)) {
dev_err(dev->dev, "Failed to allocate DRM framebuffer.\n");
ret = PTR_ERR(helper->fb);
- goto err_release_fbi;
+ goto out;
}
fbi->par = helper;
return 0;
-err_release_fbi:
- drm_fb_helper_release_fbi(helper);
-err_rockchip_gem_free_object:
+out:
rockchip_gem_free_object(&rk_obj->base);
return ret;
}
helper = &private->fbdev_helper;
drm_fb_helper_unregister_fbi(helper);
- drm_fb_helper_release_fbi(helper);
if (helper->fb)
drm_framebuffer_unreference(helper->fb);
#include <drm/drm_crtc_helper.h>
#include <drm/drm_flip_work.h>
#include <drm/drm_plane_helper.h>
+#ifdef CONFIG_DRM_ANALOGIX_DP
+#include <drm/bridge/analogix_dp.h>
+#endif
#include <linux/kernel.h>
#include <linux/module.h>
spin_unlock_irqrestore(&vop->irq_lock, flags);
}
-static const struct rockchip_crtc_funcs private_crtc_funcs = {
- .enable_vblank = vop_crtc_enable_vblank,
- .disable_vblank = vop_crtc_disable_vblank,
-};
-
static bool vop_crtc_mode_fixup(struct drm_crtc *crtc,
const struct drm_display_mode *mode,
struct drm_display_mode *adjusted_mode)
}
pin_pol = BIT(DCLK_INVERT);
- pin_pol |= (adjusted_mode->flags & DRM_MODE_FLAG_NHSYNC) ?
- 0 : BIT(HSYNC_POSITIVE);
- pin_pol |= (adjusted_mode->flags & DRM_MODE_FLAG_NVSYNC) ?
- 0 : BIT(VSYNC_POSITIVE);
+ pin_pol |= (adjusted_mode->flags & DRM_MODE_FLAG_PHSYNC) ?
+ BIT(HSYNC_POSITIVE) : 0;
+ pin_pol |= (adjusted_mode->flags & DRM_MODE_FLAG_PVSYNC) ?
+ BIT(VSYNC_POSITIVE) : 0;
VOP_CTRL_SET(vop, pin_pol, pin_pol);
switch (s->output_type) {
kfree(s);
}
+#ifdef CONFIG_DRM_ANALOGIX_DP
+static struct drm_connector *vop_get_edp_connector(struct vop *vop)
+{
+ struct drm_crtc *crtc = &vop->crtc;
+ struct drm_connector *connector;
+
+ mutex_lock(&crtc->dev->mode_config.mutex);
+ drm_for_each_connector(connector, crtc->dev)
+ if (connector->connector_type == DRM_MODE_CONNECTOR_eDP) {
+ mutex_unlock(&crtc->dev->mode_config.mutex);
+ return connector;
+ }
+ mutex_unlock(&crtc->dev->mode_config.mutex);
+
+ return NULL;
+}
+
+static int vop_crtc_set_crc_source(struct drm_crtc *crtc,
+ const char *source_name, size_t *values_cnt)
+{
+ struct vop *vop = to_vop(crtc);
+ struct drm_connector *connector;
+ int ret;
+
+ connector = vop_get_edp_connector(vop);
+ if (!connector)
+ return -EINVAL;
+
+ *values_cnt = 3;
+
+ if (source_name && strcmp(source_name, "auto") == 0)
+ ret = analogix_dp_start_crc(connector);
+ else if (!source_name)
+ ret = analogix_dp_stop_crc(connector);
+ else
+ ret = -EINVAL;
+
+ return ret;
+}
+#else
+static int vop_crtc_set_crc_source(struct drm_crtc *crtc,
+ const char *source_name, size_t *values_cnt)
+{
+ return -ENODEV;
+}
+#endif
+
static const struct drm_crtc_funcs vop_crtc_funcs = {
.set_config = drm_atomic_helper_set_config,
.page_flip = drm_atomic_helper_page_flip,
.reset = vop_crtc_reset,
.atomic_duplicate_state = vop_crtc_duplicate_state,
.atomic_destroy_state = vop_crtc_destroy_state,
+ .enable_vblank = vop_crtc_enable_vblank,
+ .disable_vblank = vop_crtc_disable_vblank,
+ .set_crc_source = vop_crtc_set_crc_source,
};
static void vop_fb_unref_worker(struct drm_flip_work *work, void *val)
init_completion(&vop->dsp_hold_completion);
init_completion(&vop->line_flag_completion);
crtc->port = port;
- rockchip_register_crtc_funcs(crtc, &private_crtc_funcs);
return 0;
struct drm_device *drm_dev = vop->drm_dev;
struct drm_plane *plane, *tmp;
- rockchip_unregister_crtc_funcs(crtc);
of_node_put(crtc->port);
/*
}
if (misalignment(node, alignment)) {
- pr_err("node is misalinged, start %llx rem %llu, expected alignment %llu\n",
+ pr_err("node is misaligned, start %llx rem %llu, expected alignment %llu\n",
node->start, misalignment(node, alignment), alignment);
ok = false;
}
n++;
}
- drm_mm_for_each_node_in_range(node, mm, 0, start) {
- if (node) {
+ if (start > 0) {
+ node = __drm_mm_interval_first(mm, 0, start - 1);
+ if (node->allocated) {
pr_err("node before start: node=%llx+%llu, start=%llx\n",
node->start, node->size, start);
return false;
}
}
- drm_mm_for_each_node_in_range(node, mm, end, U64_MAX) {
- if (node) {
+ if (end < U64_MAX) {
+ node = __drm_mm_interval_first(mm, end, U64_MAX);
+ if (node->allocated) {
pr_err("node after end: node=%llx+%llu, end=%llx\n",
node->start, node->size, end);
return false;
return 0;
}
+static void shmob_drm_crtc_enable_vblank(struct shmob_drm_device *sdev,
+ bool enable)
+{
+ unsigned long flags;
+ u32 ldintr;
+
+ /* Be careful not to acknowledge any pending interrupt. */
+ spin_lock_irqsave(&sdev->irq_lock, flags);
+ ldintr = lcdc_read(sdev, LDINTR) | LDINTR_STATUS_MASK;
+ if (enable)
+ ldintr |= LDINTR_VEE;
+ else
+ ldintr &= ~LDINTR_VEE;
+ lcdc_write(sdev, LDINTR, ldintr);
+ spin_unlock_irqrestore(&sdev->irq_lock, flags);
+}
+
+static int shmob_drm_enable_vblank(struct drm_crtc *crtc)
+{
+ struct shmob_drm_device *sdev = crtc->dev->dev_private;
+
+ shmob_drm_crtc_enable_vblank(sdev, true);
+
+ return 0;
+}
+
+static void shmob_drm_disable_vblank(struct drm_crtc *crtc)
+{
+ struct shmob_drm_device *sdev = crtc->dev->dev_private;
+
+ shmob_drm_crtc_enable_vblank(sdev, false);
+}
+
static const struct drm_crtc_funcs crtc_funcs = {
.destroy = drm_crtc_cleanup,
.set_config = drm_crtc_helper_set_config,
.page_flip = shmob_drm_crtc_page_flip,
+ .enable_vblank = shmob_drm_enable_vblank,
+ .disable_vblank = shmob_drm_disable_vblank,
};
int shmob_drm_crtc_create(struct shmob_drm_device *sdev)
return 0;
}
-void shmob_drm_crtc_enable_vblank(struct shmob_drm_device *sdev, bool enable)
-{
- unsigned long flags;
- u32 ldintr;
-
- /* Be careful not to acknowledge any pending interrupt. */
- spin_lock_irqsave(&sdev->irq_lock, flags);
- ldintr = lcdc_read(sdev, LDINTR) | LDINTR_STATUS_MASK;
- if (enable)
- ldintr |= LDINTR_VEE;
- else
- ldintr &= ~LDINTR_VEE;
- lcdc_write(sdev, LDINTR, ldintr);
- spin_unlock_irqrestore(&sdev->irq_lock, flags);
-}
-
/* -----------------------------------------------------------------------------
* Connector
*/
return ret;
drm_connector_helper_add(connector, &connector_helper_funcs);
- ret = drm_connector_register(connector);
- if (ret < 0)
- goto err_cleanup;
ret = shmob_drm_backlight_init(&sdev->connector);
if (ret < 0)
- goto err_sysfs;
+ goto err_cleanup;
ret = drm_mode_connector_attach_encoder(connector, encoder);
if (ret < 0)
err_backlight:
shmob_drm_backlight_exit(&sdev->connector);
-err_sysfs:
- drm_connector_unregister(connector);
err_cleanup:
drm_connector_cleanup(connector);
return ret;
};
int shmob_drm_crtc_create(struct shmob_drm_device *sdev);
-void shmob_drm_crtc_enable_vblank(struct shmob_drm_device *sdev, bool enable);
void shmob_drm_crtc_finish_page_flip(struct shmob_drm_crtc *scrtc);
void shmob_drm_crtc_suspend(struct shmob_drm_crtc *scrtc);
void shmob_drm_crtc_resume(struct shmob_drm_crtc *scrtc);
#include <drm/drm_crtc_helper.h>
#include <drm/drm_gem_cma_helper.h>
-#include "shmob_drm_crtc.h"
#include "shmob_drm_drv.h"
#include "shmob_drm_kms.h"
#include "shmob_drm_plane.h"
* DRM operations
*/
-static void shmob_drm_unload(struct drm_device *dev)
-{
- drm_kms_helper_poll_fini(dev);
- drm_mode_config_cleanup(dev);
- drm_vblank_cleanup(dev);
- drm_irq_uninstall(dev);
-
- dev->dev_private = NULL;
-}
-
-static int shmob_drm_load(struct drm_device *dev, unsigned long flags)
-{
- struct shmob_drm_platform_data *pdata = dev->dev->platform_data;
- struct platform_device *pdev = dev->platformdev;
- struct shmob_drm_device *sdev;
- struct resource *res;
- unsigned int i;
- int ret;
-
- if (pdata == NULL) {
- dev_err(dev->dev, "no platform data\n");
- return -EINVAL;
- }
-
- sdev = devm_kzalloc(&pdev->dev, sizeof(*sdev), GFP_KERNEL);
- if (sdev == NULL) {
- dev_err(dev->dev, "failed to allocate private data\n");
- return -ENOMEM;
- }
-
- sdev->dev = &pdev->dev;
- sdev->pdata = pdata;
- spin_lock_init(&sdev->irq_lock);
-
- sdev->ddev = dev;
- dev->dev_private = sdev;
-
- /* I/O resources and clocks */
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (res == NULL) {
- dev_err(&pdev->dev, "failed to get memory resource\n");
- return -EINVAL;
- }
-
- sdev->mmio = devm_ioremap_nocache(&pdev->dev, res->start,
- resource_size(res));
- if (sdev->mmio == NULL) {
- dev_err(&pdev->dev, "failed to remap memory resource\n");
- return -ENOMEM;
- }
-
- ret = shmob_drm_setup_clocks(sdev, pdata->clk_source);
- if (ret < 0)
- return ret;
-
- ret = shmob_drm_init_interface(sdev);
- if (ret < 0)
- return ret;
-
- ret = shmob_drm_modeset_init(sdev);
- if (ret < 0) {
- dev_err(&pdev->dev, "failed to initialize mode setting\n");
- return ret;
- }
-
- for (i = 0; i < 4; ++i) {
- ret = shmob_drm_plane_create(sdev, i);
- if (ret < 0) {
- dev_err(&pdev->dev, "failed to create plane %u\n", i);
- goto done;
- }
- }
-
- ret = drm_vblank_init(dev, 1);
- if (ret < 0) {
- dev_err(&pdev->dev, "failed to initialize vblank\n");
- goto done;
- }
-
- ret = drm_irq_install(dev, platform_get_irq(dev->platformdev, 0));
- if (ret < 0) {
- dev_err(&pdev->dev, "failed to install IRQ handler\n");
- goto done;
- }
-
- platform_set_drvdata(pdev, sdev);
-
-done:
- if (ret)
- shmob_drm_unload(dev);
-
- return ret;
-}
-
static irqreturn_t shmob_drm_irq(int irq, void *arg)
{
struct drm_device *dev = arg;
return IRQ_HANDLED;
}
-static int shmob_drm_enable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- struct shmob_drm_device *sdev = dev->dev_private;
-
- shmob_drm_crtc_enable_vblank(sdev, true);
-
- return 0;
-}
-
-static void shmob_drm_disable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- struct shmob_drm_device *sdev = dev->dev_private;
-
- shmob_drm_crtc_enable_vblank(sdev, false);
-}
-
-static const struct file_operations shmob_drm_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .poll = drm_poll,
- .read = drm_read,
- .llseek = no_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(shmob_drm_fops);
static struct drm_driver shmob_drm_driver = {
.driver_features = DRIVER_HAVE_IRQ | DRIVER_GEM | DRIVER_MODESET
| DRIVER_PRIME,
- .load = shmob_drm_load,
- .unload = shmob_drm_unload,
.irq_handler = shmob_drm_irq,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = shmob_drm_enable_vblank,
- .disable_vblank = shmob_drm_disable_vblank,
.gem_free_object_unlocked = drm_gem_cma_free_object,
.gem_vm_ops = &drm_gem_cma_vm_ops,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
* Platform driver
*/
-static int shmob_drm_probe(struct platform_device *pdev)
+static int shmob_drm_remove(struct platform_device *pdev)
{
- return drm_platform_init(&shmob_drm_driver, pdev);
+ struct shmob_drm_device *sdev = platform_get_drvdata(pdev);
+ struct drm_device *ddev = sdev->ddev;
+
+ drm_dev_unregister(ddev);
+ drm_kms_helper_poll_fini(ddev);
+ drm_mode_config_cleanup(ddev);
+ drm_irq_uninstall(ddev);
+ drm_dev_unref(ddev);
+
+ return 0;
}
-static int shmob_drm_remove(struct platform_device *pdev)
+static int shmob_drm_probe(struct platform_device *pdev)
{
- struct shmob_drm_device *sdev = platform_get_drvdata(pdev);
+ struct shmob_drm_platform_data *pdata = pdev->dev.platform_data;
+ struct shmob_drm_device *sdev;
+ struct drm_device *ddev;
+ struct resource *res;
+ unsigned int i;
+ int ret;
+
+ if (pdata == NULL) {
+ dev_err(&pdev->dev, "no platform data\n");
+ return -EINVAL;
+ }
+
+ /*
+ * Allocate and initialize the driver private data, I/O resources and
+ * clocks.
+ */
+ sdev = devm_kzalloc(&pdev->dev, sizeof(*sdev), GFP_KERNEL);
+ if (sdev == NULL)
+ return -ENOMEM;
- drm_put_dev(sdev->ddev);
+ sdev->dev = &pdev->dev;
+ sdev->pdata = pdata;
+ spin_lock_init(&sdev->irq_lock);
+
+ platform_set_drvdata(pdev, sdev);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ sdev->mmio = devm_ioremap_resource(&pdev->dev, res);
+ if (sdev->mmio == NULL)
+ return -ENOMEM;
+
+ ret = shmob_drm_setup_clocks(sdev, pdata->clk_source);
+ if (ret < 0)
+ return ret;
+
+ ret = shmob_drm_init_interface(sdev);
+ if (ret < 0)
+ return ret;
+
+ /* Allocate and initialize the DRM device. */
+ ddev = drm_dev_alloc(&shmob_drm_driver, &pdev->dev);
+ if (IS_ERR(ddev))
+ return PTR_ERR(ddev);
+
+ sdev->ddev = ddev;
+ ddev->dev_private = sdev;
+
+ ret = shmob_drm_modeset_init(sdev);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "failed to initialize mode setting\n");
+ goto err_free_drm_dev;
+ }
+
+ for (i = 0; i < 4; ++i) {
+ ret = shmob_drm_plane_create(sdev, i);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "failed to create plane %u\n", i);
+ goto err_modeset_cleanup;
+ }
+ }
+
+ ret = drm_vblank_init(ddev, 1);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "failed to initialize vblank\n");
+ goto err_modeset_cleanup;
+ }
+
+ ret = drm_irq_install(ddev, platform_get_irq(pdev, 0));
+ if (ret < 0) {
+ dev_err(&pdev->dev, "failed to install IRQ handler\n");
+ goto err_vblank_cleanup;
+ }
+
+ /*
+ * Register the DRM device with the core and the connectors with
+ * sysfs.
+ */
+ ret = drm_dev_register(ddev, 0);
+ if (ret < 0)
+ goto err_irq_uninstall;
return 0;
+
+err_irq_uninstall:
+ drm_irq_uninstall(ddev);
+err_vblank_cleanup:
+ drm_vblank_cleanup(ddev);
+err_modeset_cleanup:
+ drm_kms_helper_poll_fini(ddev);
+ drm_mode_config_cleanup(ddev);
+err_free_drm_dev:
+ drm_dev_unref(ddev);
+
+ return ret;
}
static struct platform_driver shmob_drm_platform_driver = {
dev->mode_config.funcs = &sti_mode_config_funcs;
}
-static const struct file_operations sti_driver_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .mmap = drm_gem_cma_mmap,
- .poll = drm_poll,
- .read = drm_read,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .release = drm_release,
-};
+DEFINE_DRM_GEM_CMA_FOPS(sti_driver_fops);
static struct drm_driver sti_driver = {
.driver_features = DRIVER_MODESET |
.dumb_destroy = drm_gem_dumb_destroy,
.fops = &sti_driver_fops,
- .get_vblank_counter = drm_vblank_no_hw_counter,
.enable_vblank = sti_crtc_enable_vblank,
.disable_vblank = sti_crtc_disable_vblank,
if (IS_ERR(ddev))
return PTR_ERR(ddev);
- ddev->platformdev = to_platform_device(dev);
-
ret = sti_init(ddev);
if (ret)
goto err_drm_dev_unref;
dma_set_coherent_mask(dev, DMA_BIT_MASK(32));
- of_platform_populate(node, NULL, NULL, dev);
+ devm_of_platform_populate(dev);
child_np = of_get_next_available_child(node, NULL);
static int sti_platform_remove(struct platform_device *pdev)
{
component_master_del(&pdev->dev, &sti_ops);
- of_platform_depopulate(&pdev->dev);
return 0;
}
.enable = sun4i_crtc_enable,
};
+static int sun4i_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ struct sun4i_crtc *scrtc = drm_crtc_to_sun4i_crtc(crtc);
+ struct sun4i_drv *drv = scrtc->drv;
+
+ DRM_DEBUG_DRIVER("Enabling VBLANK on crtc %p\n", crtc);
+
+ sun4i_tcon_enable_vblank(drv->tcon, true);
+
+ return 0;
+}
+
+static void sun4i_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+ struct sun4i_crtc *scrtc = drm_crtc_to_sun4i_crtc(crtc);
+ struct sun4i_drv *drv = scrtc->drv;
+
+ DRM_DEBUG_DRIVER("Disabling VBLANK on crtc %p\n", crtc);
+
+ sun4i_tcon_enable_vblank(drv->tcon, false);
+}
+
static const struct drm_crtc_funcs sun4i_crtc_funcs = {
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
.atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
.page_flip = drm_atomic_helper_page_flip,
.reset = drm_atomic_helper_crtc_reset,
.set_config = drm_atomic_helper_set_config,
+ .enable_vblank = sun4i_crtc_enable_vblank,
+ .disable_vblank = sun4i_crtc_disable_vblank,
};
struct sun4i_crtc *sun4i_crtc_init(struct drm_device *drm)
#include "sun4i_drv.h"
#include "sun4i_framebuffer.h"
#include "sun4i_layer.h"
-#include "sun4i_tcon.h"
-static int sun4i_drv_enable_vblank(struct drm_device *drm, unsigned int pipe)
-{
- struct sun4i_drv *drv = drm->dev_private;
- struct sun4i_tcon *tcon = drv->tcon;
-
- DRM_DEBUG_DRIVER("Enabling VBLANK on pipe %d\n", pipe);
-
- sun4i_tcon_enable_vblank(tcon, true);
-
- return 0;
-}
-
-static void sun4i_drv_disable_vblank(struct drm_device *drm, unsigned int pipe)
-{
- struct sun4i_drv *drv = drm->dev_private;
- struct sun4i_tcon *tcon = drv->tcon;
-
- DRM_DEBUG_DRIVER("Disabling VBLANK on pipe %d\n", pipe);
-
- sun4i_tcon_enable_vblank(tcon, false);
-}
-
-static const struct file_operations sun4i_drv_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .poll = drm_poll,
- .read = drm_read,
- .llseek = no_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(sun4i_drv_fops);
static struct drm_driver sun4i_drv_driver = {
.driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_PRIME | DRIVER_ATOMIC,
.gem_prime_mmap = drm_gem_cma_prime_mmap,
/* Frame Buffer Operations */
-
- /* VBlank Operations */
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = sun4i_drv_enable_vblank,
- .disable_vblank = sun4i_drv_disable_vblank,
};
static void sun4i_remove_framebuffers(void)
return 0;
}
-u32 tegra_dc_get_vblank_counter(struct tegra_dc *dc)
+static u32 tegra_dc_get_vblank_counter(struct drm_crtc *crtc)
{
+ struct tegra_dc *dc = to_tegra_dc(crtc);
+
if (dc->syncpt)
return host1x_syncpt_read(dc->syncpt);
return drm_crtc_vblank_count(&dc->base);
}
-void tegra_dc_enable_vblank(struct tegra_dc *dc)
+static int tegra_dc_enable_vblank(struct drm_crtc *crtc)
{
+ struct tegra_dc *dc = to_tegra_dc(crtc);
unsigned long value, flags;
spin_lock_irqsave(&dc->lock, flags);
tegra_dc_writel(dc, value, DC_CMD_INT_MASK);
spin_unlock_irqrestore(&dc->lock, flags);
+
+ return 0;
}
-void tegra_dc_disable_vblank(struct tegra_dc *dc)
+static void tegra_dc_disable_vblank(struct drm_crtc *crtc)
{
+ struct tegra_dc *dc = to_tegra_dc(crtc);
unsigned long value, flags;
spin_lock_irqsave(&dc->lock, flags);
.reset = tegra_crtc_reset,
.atomic_duplicate_state = tegra_crtc_atomic_duplicate_state,
.atomic_destroy_state = tegra_crtc_atomic_destroy_state,
+ .get_vblank_counter = tegra_dc_get_vblank_counter,
+ .enable_vblank = tegra_dc_enable_vblank,
+ .disable_vblank = tegra_dc_disable_vblank,
};
static int tegra_dc_set_timings(struct tegra_dc *dc,
.llseek = noop_llseek,
};
-static u32 tegra_drm_get_vblank_counter(struct drm_device *drm,
- unsigned int pipe)
-{
- struct drm_crtc *crtc = drm_crtc_from_index(drm, pipe);
- struct tegra_dc *dc = to_tegra_dc(crtc);
-
- if (!crtc)
- return 0;
-
- return tegra_dc_get_vblank_counter(dc);
-}
-
-static int tegra_drm_enable_vblank(struct drm_device *drm, unsigned int pipe)
-{
- struct drm_crtc *crtc = drm_crtc_from_index(drm, pipe);
- struct tegra_dc *dc = to_tegra_dc(crtc);
-
- if (!crtc)
- return -ENODEV;
-
- tegra_dc_enable_vblank(dc);
-
- return 0;
-}
-
-static void tegra_drm_disable_vblank(struct drm_device *drm, unsigned int pipe)
-{
- struct drm_crtc *crtc = drm_crtc_from_index(drm, pipe);
- struct tegra_dc *dc = to_tegra_dc(crtc);
-
- if (crtc)
- tegra_dc_disable_vblank(dc);
-}
-
static void tegra_drm_preclose(struct drm_device *drm, struct drm_file *file)
{
struct tegra_drm_file *fpriv = file->driver_priv;
.preclose = tegra_drm_preclose,
.lastclose = tegra_drm_lastclose,
- .get_vblank_counter = tegra_drm_get_vblank_counter,
- .enable_vblank = tegra_drm_enable_vblank,
- .disable_vblank = tegra_drm_disable_vblank,
-
#if defined(CONFIG_DEBUG_FS)
.debugfs_init = tegra_debugfs_init,
#endif
};
/* from dc.c */
-u32 tegra_dc_get_vblank_counter(struct tegra_dc *dc);
-void tegra_dc_enable_vblank(struct tegra_dc *dc);
-void tegra_dc_disable_vblank(struct tegra_dc *dc);
void tegra_dc_commit(struct tegra_dc *dc);
int tegra_dc_state_setup_clock(struct tegra_dc *dc,
struct drm_crtc_state *crtc_state,
dev_err(drm->dev, "failed to allocate DRM framebuffer: %d\n",
err);
drm_gem_object_unreference_unlocked(&bo->gem);
- goto release;
+ return PTR_ERR(fbdev->fb);
}
fb = &fbdev->fb->base;
destroy:
drm_framebuffer_remove(fb);
-release:
- drm_fb_helper_release_fbi(helper);
return err;
}
static void tegra_fbdev_exit(struct tegra_fbdev *fbdev)
{
drm_fb_helper_unregister_fbi(&fbdev->base);
- drm_fb_helper_release_fbi(&fbdev->base);
if (fbdev->fb)
drm_framebuffer_remove(&fbdev->fb->base);
return 0;
}
+static int tilcdc_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ return 0;
+}
+
+static void tilcdc_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+}
+
static const struct drm_crtc_funcs tilcdc_crtc_funcs = {
.destroy = tilcdc_crtc_destroy,
.set_config = drm_atomic_helper_set_config,
.reset = drm_atomic_helper_crtc_reset,
.atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .enable_vblank = tilcdc_crtc_enable_vblank,
+ .disable_vblank = tilcdc_crtc_disable_vblank,
};
static const struct drm_crtc_helper_funcs tilcdc_crtc_helper_funcs = {
if (IS_ERR(ddev))
return PTR_ERR(ddev);
- ddev->platformdev = pdev;
ddev->dev_private = priv;
platform_set_drvdata(pdev, ddev);
drm_mode_config_init(ddev);
return tilcdc_crtc_irq(priv->crtc);
}
-static int tilcdc_enable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- return 0;
-}
-
-static void tilcdc_disable_vblank(struct drm_device *dev, unsigned int pipe)
-{
- return;
-}
-
#if defined(CONFIG_DEBUG_FS)
static const struct {
const char *name;
}
#endif
-static const struct file_operations fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
- .compat_ioctl = drm_compat_ioctl,
- .poll = drm_poll,
- .read = drm_read,
- .llseek = no_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(fops);
static struct drm_driver tilcdc_driver = {
.driver_features = (DRIVER_HAVE_IRQ | DRIVER_GEM | DRIVER_MODESET |
DRIVER_PRIME | DRIVER_ATOMIC),
.lastclose = tilcdc_lastclose,
.irq_handler = tilcdc_irq,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = tilcdc_enable_vblank,
- .disable_vblank = tilcdc_disable_vblank,
.gem_free_object_unlocked = drm_gem_cma_free_object,
.gem_vm_ops = &drm_gem_cma_vm_ops,
.dumb_create = drm_gem_cma_dumb_create,
ret = spi_sync(spi, &m);
if (ret)
return ret;
- };
+ }
return 0;
}
ret = spi_sync(spi, &m);
if (ret)
return ret;
- };
+ }
return 0;
}
ret = spi_sync(spi, &m);
if (ret)
return ret;
- };
+ }
return 0;
}
}
if (!type_found) {
- printk(KERN_ERR TTM_PFX "No compatible memory type found.\n");
+ pr_err(TTM_PFX "No compatible memory type found\n");
return -EINVAL;
}
ret = udl_framebuffer_init(dev, &ufbdev->ufb, &mode_cmd, obj);
if (ret)
- goto out_destroy_fbi;
+ goto out_gfree;
fb = &ufbdev->ufb.base;
ufbdev->ufb.obj->vmapping);
return ret;
-out_destroy_fbi:
- drm_fb_helper_release_fbi(helper);
out_gfree:
drm_gem_object_unreference_unlocked(&ufbdev->ufb.obj->base);
out:
struct udl_fbdev *ufbdev)
{
drm_fb_helper_unregister_fbi(&ufbdev->helper);
- drm_fb_helper_release_fbi(&ufbdev->helper);
drm_fb_helper_fini(&ufbdev->helper);
drm_framebuffer_unregister_private(&ufbdev->ufb.base);
drm_framebuffer_cleanup(&ufbdev->ufb.base);
tristate "Broadcom VC4 Graphics"
depends on ARCH_BCM2835 || COMPILE_TEST
depends on DRM
+ depends on SND && SND_SOC
depends on COMMON_CLK
select DRM_KMS_HELPER
select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
select DRM_PANEL
+ select SND_PCM
+ select SND_PCM_ELD
+ select SND_SOC_GENERIC_DMAENGINE_PCM
select DRM_MIPI_DSI
help
Choose this option if you have a system that has a Broadcom
* published by the Free Software Foundation.
*/
-/* DOC: VC4 GEM BO management support.
+/**
+ * DOC: VC4 GEM BO management support
*
* The VC4 GPU architecture (both scanout and rendering) has direct
* access to system memory with no MMU in between. To support it, we
/**
* vc4_gem_create_object - Implementation of driver->gem_create_object.
+ * @dev: DRM device
+ * @size: Size in bytes of the memory the object will reference
*
* This lets the CMA helpers allocate object structs for us, and keep
* our BO stats correct.
}
struct vc4_bo *vc4_bo_create(struct drm_device *dev, size_t unaligned_size,
- bool from_cache)
+ bool allow_unzeroed)
{
size_t size = roundup(unaligned_size, PAGE_SIZE);
struct vc4_dev *vc4 = to_vc4_dev(dev);
struct drm_gem_cma_object *cma_obj;
+ struct vc4_bo *bo;
if (size == 0)
return ERR_PTR(-EINVAL);
/* First, try to get a vc4_bo from the kernel BO cache. */
- if (from_cache) {
- struct vc4_bo *bo = vc4_bo_get_from_cache(dev, size);
-
- if (bo)
- return bo;
+ bo = vc4_bo_get_from_cache(dev, size);
+ if (bo) {
+ if (!allow_unzeroed)
+ memset(bo->base.vaddr, 0, bo->base.base.size);
+ return bo;
}
cma_obj = drm_gem_cma_create(dev, size);
goto out;
}
+ /* If this object was partially constructed but CMA allocation
+ * had failed, just free it.
+ */
+ if (!bo->base.vaddr) {
+ vc4_bo_destroy(bo);
+ goto out;
+ }
+
cache_list = vc4_get_cache_list_for_size(dev, gem_bo->size);
if (!cache_list) {
vc4_bo_destroy(bo);
*
* In VC4, the Pixel Valve is what most closely corresponds to the
* DRM's concept of a CRTC. The PV generates video timings from the
- * output's clock plus its configuration. It pulls scaled pixels from
+ * encoder's clock plus its configuration. It pulls scaled pixels from
* the HVS at that timing, and feeds it to the encoder.
*
* However, the DRM CRTC also collects the configuration of all the
- * DRM planes attached to it. As a result, this file also manages
- * setup of the VC4 HVS's display elements on the CRTC.
+ * DRM planes attached to it. As a result, the CRTC is also
+ * responsible for writing the display list for the HVS channel that
+ * the CRTC will use.
*
* The 2835 has 3 different pixel valves. pv0 in the audio power
* domain feeds DSI0 or DPI, while pv1 feeds DS1 or SMI. pv2 in the
}
}
-int vc4_enable_vblank(struct drm_device *dev, unsigned int crtc_id)
+static int vc4_enable_vblank(struct drm_crtc *crtc)
{
- struct drm_crtc *crtc = drm_crtc_from_index(dev, crtc_id);
struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
CRTC_WRITE(PV_INTEN, PV_INT_VFP_START);
return 0;
}
-void vc4_disable_vblank(struct drm_device *dev, unsigned int crtc_id)
+static void vc4_disable_vblank(struct drm_crtc *crtc)
{
- struct drm_crtc *crtc = drm_crtc_from_index(dev, crtc_id);
struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
CRTC_WRITE(PV_INTEN, 0);
.atomic_duplicate_state = vc4_crtc_duplicate_state,
.atomic_destroy_state = vc4_crtc_destroy_state,
.gamma_set = vc4_crtc_gamma_set,
+ .enable_vblank = vc4_enable_vblank,
+ .disable_vblank = vc4_disable_vblank,
};
static const struct drm_crtc_helper_funcs vc4_crtc_helper_funcs = {
* DOC: VC4 DPI module
*
* The VC4 DPI hardware supports MIPI DPI type 4 and Nokia ViSSI
- * signals, which are routed out to GPIO0-27 with the ALT2 function.
+ * signals. On BCM2835, these can be routed out to GPIO0-27 with the
+ * ALT2 function.
*/
#include "drm_atomic_helper.h"
DPI_REG(DPI_ID),
};
-static void vc4_dpi_dump_regs(struct vc4_dpi *dpi)
-{
- int i;
-
- for (i = 0; i < ARRAY_SIZE(dpi_regs); i++) {
- DRM_INFO("0x%04x (%s): 0x%08x\n",
- dpi_regs[i].reg, dpi_regs[i].name,
- DPI_READ(dpi_regs[i].reg));
- }
-}
-
#ifdef CONFIG_DEBUG_FS
int vc4_dpi_debugfs_regs(struct seq_file *m, void *unused)
{
if (IS_ERR(dpi->regs))
return PTR_ERR(dpi->regs);
- vc4_dpi_dump_regs(dpi);
-
if (DPI_READ(DPI_ID) != DPI_ID_VALUE) {
dev_err(dev, "Port returned 0x%08x for ID instead of 0x%08x\n",
DPI_READ(DPI_ID), DPI_ID_VALUE);
* published by the Free Software Foundation.
*/
+/**
+ * DOC: Broadcom VC4 Graphics Driver
+ *
+ * The Broadcom VideoCore 4 (present in the Raspberry Pi) contains a
+ * OpenGL ES 2.0-compatible 3D engine called V3D, and a highly
+ * configurable display output pipeline that supports HDMI, DSI, DPI,
+ * and Composite TV output.
+ *
+ * The 3D engine also has an interface for submitting arbitrary
+ * compute shader-style jobs using the same shader processor as is
+ * used for vertex and fragment shaders in GLES 2.0. However, given
+ * that the hardware isn't able to expose any standard interfaces like
+ * OpenGL compute shaders or OpenCL, it isn't supported by this
+ * driver.
+ */
+
#include <linux/clk.h>
#include <linux/component.h>
#include <linux/device.h>
.irq_postinstall = vc4_irq_postinstall,
.irq_uninstall = vc4_irq_uninstall,
- .enable_vblank = vc4_enable_vblank,
- .disable_vblank = vc4_disable_vblank,
- .get_vblank_counter = drm_vblank_no_hw_counter,
.get_scanout_position = vc4_crtc_get_scanoutpos,
.get_vblank_timestamp = vc4_crtc_get_vblank_timestamp,
static int __init vc4_drm_register(void)
{
- int i, ret;
+ int ret;
+
+ ret = platform_register_drivers(component_drivers,
+ ARRAY_SIZE(component_drivers));
+ if (ret)
+ return ret;
- for (i = 0; i < ARRAY_SIZE(component_drivers); i++) {
- ret = platform_driver_register(component_drivers[i]);
- if (ret) {
- while (--i >= 0)
- platform_driver_unregister(component_drivers[i]);
- return ret;
- }
- }
return platform_driver_register(&vc4_platform_driver);
}
static void __exit vc4_drm_unregister(void)
{
- int i;
-
- for (i = ARRAY_SIZE(component_drivers) - 1; i >= 0; i--)
- platform_driver_unregister(component_drivers[i]);
-
+ platform_unregister_drivers(component_drivers,
+ ARRAY_SIZE(component_drivers));
platform_driver_unregister(&vc4_platform_driver);
}
/* vc4_crtc.c */
extern struct platform_driver vc4_crtc_driver;
-int vc4_enable_vblank(struct drm_device *dev, unsigned int crtc_id);
-void vc4_disable_vblank(struct drm_device *dev, unsigned int crtc_id);
bool vc4_event_pending(struct drm_crtc *crtc);
int vc4_crtc_debugfs_regs(struct seq_file *m, void *arg);
int vc4_crtc_get_scanoutpos(struct drm_device *dev, unsigned int crtc_id,
static struct drm_connector *vc4_dsi_connector_init(struct drm_device *dev,
struct vc4_dsi *dsi)
{
- struct drm_connector *connector = NULL;
+ struct drm_connector *connector;
struct vc4_dsi_connector *dsi_connector;
- int ret = 0;
dsi_connector = devm_kzalloc(dev->dev, sizeof(*dsi_connector),
GFP_KERNEL);
- if (!dsi_connector) {
- ret = -ENOMEM;
- goto fail;
- }
+ if (!dsi_connector)
+ return ERR_PTR(-ENOMEM);
+
connector = &dsi_connector->base;
dsi_connector->dsi = dsi;
drm_mode_connector_attach_encoder(connector, dsi->encoder);
return connector;
-
-fail:
- if (connector)
- vc4_dsi_connector_destroy(connector);
-
- return ERR_PTR(ret);
}
static void vc4_dsi_encoder_destroy(struct drm_encoder *encoder)
}
/**
- * Exposes clocks generated by the analog PHY that are consumed by
- * CPRMAN (clk-bcm2835.c).
+ * vc4_dsi_init_phy_clocks - Exposes clocks generated by the analog
+ * PHY that are consumed by CPRMAN (clk-bcm2835.c).
+ * @dsi: DSI encoder
*/
static int
vc4_dsi_init_phy_clocks(struct vc4_dsi *dsi)
}
/**
- * Looks up a bunch of GEM handles for BOs and stores the array for
- * use in the command validator that actually writes relocated
- * addresses pointing to them.
+ * vc4_cl_lookup_bos() - Sets up exec->bo[] with the GEM objects
+ * referenced by the job.
+ * @dev: DRM device
+ * @file_priv: DRM file for this fd
+ * @exec: V3D job being set up
+ *
+ * The command validator needs to reference BOs by their index within
+ * the submitted job's BO list. This does the validation of the job's
+ * BO list and reference counting for the lifetime of the job.
+ *
+ * Note that this function doesn't need to unreference the BOs on
+ * failure, because that will happen at vc4_complete_exec() time.
*/
static int
vc4_cl_lookup_bos(struct drm_device *dev,
}
/**
- * Submits a command list to the VC4.
+ * vc4_submit_cl_ioctl() - Submits a job (frame) to the VC4.
+ * @dev: DRM device
+ * @data: ioctl argument
+ * @file_priv: DRM file for this fd
*
- * This is what is called batchbuffer emitting on other hardware.
+ * This is the main entrypoint for userspace to submit a 3D frame to
+ * the GPU. Userspace provides the binner command list (if
+ * applicable), and the kernel sets up the render command list to draw
+ * to the framebuffer described in the ioctl, using the command lists
+ * that the 3D engine's binner will produce.
*/
int
vc4_submit_cl_ioctl(struct drm_device *dev, void *data,
/**
* DOC: VC4 Falcon HDMI module
*
- * The HDMI core has a state machine and a PHY. Most of the unit
- * operates off of the HSM clock from CPRMAN. It also internally uses
- * the PLLH_PIX clock for the PHY.
+ * The HDMI core has a state machine and a PHY. On BCM2835, most of
+ * the unit operates off of the HSM clock from CPRMAN. It also
+ * internally uses the PLLH_PIX clock for the PHY.
+ *
+ * HDMI infoframes are kept within a small packet ram, where each
+ * packet can be individually enabled for including in a frame.
+ *
+ * HDMI audio is implemented entirely within the HDMI IP block. A
+ * register in the HDMI encoder takes SPDIF frames from the DMA engine
+ * and transfers them over an internal MAI (multi-channel audio
+ * interconnect) bus to the encoder side for insertion into the video
+ * blank regions.
+ *
+ * The driver's HDMI encoder does not yet support power management.
+ * The HDMI encoder's power domain and the HSM/pixel clocks are kept
+ * continuously running, and only the HDMI logic and packet ram are
+ * powered off/on at disable/enable time.
+ *
+ * The driver does not yet support CEC control, though the HDMI
+ * encoder block has CEC support.
*/
#include "drm_atomic_helper.h"
#include "linux/clk.h"
#include "linux/component.h"
#include "linux/i2c.h"
+#include "linux/of_address.h"
#include "linux/of_gpio.h"
#include "linux/of_platform.h"
+#include "linux/rational.h"
+#include "sound/dmaengine_pcm.h"
+#include "sound/pcm_drm_eld.h"
+#include "sound/pcm_params.h"
+#include "sound/soc.h"
#include "vc4_drv.h"
#include "vc4_regs.h"
+/* HDMI audio information */
+struct vc4_hdmi_audio {
+ struct snd_soc_card card;
+ struct snd_soc_dai_link link;
+ int samplerate;
+ int channels;
+ struct snd_dmaengine_dai_dma_data dma_data;
+ struct snd_pcm_substream *substream;
+};
+
/* General HDMI hardware state. */
struct vc4_hdmi {
struct platform_device *pdev;
struct drm_encoder *encoder;
struct drm_connector *connector;
+ struct vc4_hdmi_audio audio;
+
struct i2c_adapter *ddc;
void __iomem *hdmicore_regs;
void __iomem *hd_regs;
HDMI_REG(VC4_HDMI_SW_RESET_CONTROL),
HDMI_REG(VC4_HDMI_HOTPLUG_INT),
HDMI_REG(VC4_HDMI_HOTPLUG),
+ HDMI_REG(VC4_HDMI_MAI_CHANNEL_MAP),
+ HDMI_REG(VC4_HDMI_MAI_CONFIG),
+ HDMI_REG(VC4_HDMI_MAI_FORMAT),
+ HDMI_REG(VC4_HDMI_AUDIO_PACKET_CONFIG),
HDMI_REG(VC4_HDMI_RAM_PACKET_CONFIG),
HDMI_REG(VC4_HDMI_HORZA),
HDMI_REG(VC4_HDMI_HORZB),
HDMI_REG(VC4_HDMI_VERTB0),
HDMI_REG(VC4_HDMI_VERTB1),
HDMI_REG(VC4_HDMI_TX_PHY_RESET_CTL),
+ HDMI_REG(VC4_HDMI_TX_PHY_CTL0),
};
static const struct {
} hd_regs[] = {
HDMI_REG(VC4_HD_M_CTL),
HDMI_REG(VC4_HD_MAI_CTL),
+ HDMI_REG(VC4_HD_MAI_THR),
+ HDMI_REG(VC4_HD_MAI_FMT),
+ HDMI_REG(VC4_HD_MAI_SMP),
HDMI_REG(VC4_HD_VID_CTL),
HDMI_REG(VC4_HD_CSC_CTL),
HDMI_REG(VC4_HD_FRAME_COUNT),
drm_mode_connector_update_edid_property(connector, edid);
ret = drm_add_edid_modes(connector, edid);
+ drm_edid_to_eld(connector, edid);
return ret;
}
struct drm_device *dev = encoder->dev;
struct vc4_dev *vc4 = to_vc4_dev(dev);
u32 packet_id = frame->any.type - 0x80;
- u32 packet_reg = VC4_HDMI_GCP_0 + VC4_HDMI_PACKET_STRIDE * packet_id;
+ u32 packet_reg = VC4_HDMI_RAM_PACKET(packet_id);
uint8_t buffer[VC4_HDMI_PACKET_STRIDE];
ssize_t len, i;
int ret;
vc4_hdmi_write_infoframe(encoder, &frame);
}
+static void vc4_hdmi_set_audio_infoframe(struct drm_encoder *encoder)
+{
+ struct drm_device *drm = encoder->dev;
+ struct vc4_dev *vc4 = drm->dev_private;
+ struct vc4_hdmi *hdmi = vc4->hdmi;
+ union hdmi_infoframe frame;
+ int ret;
+
+ ret = hdmi_audio_infoframe_init(&frame.audio);
+
+ frame.audio.coding_type = HDMI_AUDIO_CODING_TYPE_STREAM;
+ frame.audio.sample_frequency = HDMI_AUDIO_SAMPLE_FREQUENCY_STREAM;
+ frame.audio.sample_size = HDMI_AUDIO_SAMPLE_SIZE_STREAM;
+ frame.audio.channels = hdmi->audio.channels;
+
+ vc4_hdmi_write_infoframe(encoder, &frame);
+}
+
static void vc4_hdmi_set_infoframes(struct drm_encoder *encoder)
{
vc4_hdmi_set_avi_infoframe(encoder);
.enable = vc4_hdmi_encoder_enable,
};
+/* HDMI audio codec callbacks */
+static void vc4_hdmi_audio_set_mai_clock(struct vc4_hdmi *hdmi)
+{
+ struct drm_device *drm = hdmi->encoder->dev;
+ struct vc4_dev *vc4 = to_vc4_dev(drm);
+ u32 hsm_clock = clk_get_rate(hdmi->hsm_clock);
+ unsigned long n, m;
+
+ rational_best_approximation(hsm_clock, hdmi->audio.samplerate,
+ VC4_HD_MAI_SMP_N_MASK >>
+ VC4_HD_MAI_SMP_N_SHIFT,
+ (VC4_HD_MAI_SMP_M_MASK >>
+ VC4_HD_MAI_SMP_M_SHIFT) + 1,
+ &n, &m);
+
+ HD_WRITE(VC4_HD_MAI_SMP,
+ VC4_SET_FIELD(n, VC4_HD_MAI_SMP_N) |
+ VC4_SET_FIELD(m - 1, VC4_HD_MAI_SMP_M));
+}
+
+static void vc4_hdmi_set_n_cts(struct vc4_hdmi *hdmi)
+{
+ struct drm_encoder *encoder = hdmi->encoder;
+ struct drm_crtc *crtc = encoder->crtc;
+ struct drm_device *drm = encoder->dev;
+ struct vc4_dev *vc4 = to_vc4_dev(drm);
+ const struct drm_display_mode *mode = &crtc->state->adjusted_mode;
+ u32 samplerate = hdmi->audio.samplerate;
+ u32 n, cts;
+ u64 tmp;
+
+ n = 128 * samplerate / 1000;
+ tmp = (u64)(mode->clock * 1000) * n;
+ do_div(tmp, 128 * samplerate);
+ cts = tmp;
+
+ HDMI_WRITE(VC4_HDMI_CRP_CFG,
+ VC4_HDMI_CRP_CFG_EXTERNAL_CTS_EN |
+ VC4_SET_FIELD(n, VC4_HDMI_CRP_CFG_N));
+
+ /*
+ * We could get slightly more accurate clocks in some cases by
+ * providing a CTS_1 value. The two CTS values are alternated
+ * between based on the period fields
+ */
+ HDMI_WRITE(VC4_HDMI_CTS_0, cts);
+ HDMI_WRITE(VC4_HDMI_CTS_1, cts);
+}
+
+static inline struct vc4_hdmi *dai_to_hdmi(struct snd_soc_dai *dai)
+{
+ struct snd_soc_card *card = snd_soc_dai_get_drvdata(dai);
+
+ return snd_soc_card_get_drvdata(card);
+}
+
+static int vc4_hdmi_audio_startup(struct snd_pcm_substream *substream,
+ struct snd_soc_dai *dai)
+{
+ struct vc4_hdmi *hdmi = dai_to_hdmi(dai);
+ struct drm_encoder *encoder = hdmi->encoder;
+ struct vc4_dev *vc4 = to_vc4_dev(encoder->dev);
+ int ret;
+
+ if (hdmi->audio.substream && hdmi->audio.substream != substream)
+ return -EINVAL;
+
+ hdmi->audio.substream = substream;
+
+ /*
+ * If the HDMI encoder hasn't probed, or the encoder is
+ * currently in DVI mode, treat the codec dai as missing.
+ */
+ if (!encoder->crtc || !(HDMI_READ(VC4_HDMI_RAM_PACKET_CONFIG) &
+ VC4_HDMI_RAM_PACKET_ENABLE))
+ return -ENODEV;
+
+ ret = snd_pcm_hw_constraint_eld(substream->runtime,
+ hdmi->connector->eld);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int vc4_hdmi_audio_set_fmt(struct snd_soc_dai *dai, unsigned int fmt)
+{
+ return 0;
+}
+
+static void vc4_hdmi_audio_reset(struct vc4_hdmi *hdmi)
+{
+ struct drm_encoder *encoder = hdmi->encoder;
+ struct drm_device *drm = encoder->dev;
+ struct device *dev = &hdmi->pdev->dev;
+ struct vc4_dev *vc4 = to_vc4_dev(drm);
+ int ret;
+
+ ret = vc4_hdmi_stop_packet(encoder, HDMI_INFOFRAME_TYPE_AUDIO);
+ if (ret)
+ dev_err(dev, "Failed to stop audio infoframe: %d\n", ret);
+
+ HD_WRITE(VC4_HD_MAI_CTL, VC4_HD_MAI_CTL_RESET);
+ HD_WRITE(VC4_HD_MAI_CTL, VC4_HD_MAI_CTL_ERRORF);
+ HD_WRITE(VC4_HD_MAI_CTL, VC4_HD_MAI_CTL_FLUSH);
+}
+
+static void vc4_hdmi_audio_shutdown(struct snd_pcm_substream *substream,
+ struct snd_soc_dai *dai)
+{
+ struct vc4_hdmi *hdmi = dai_to_hdmi(dai);
+
+ if (substream != hdmi->audio.substream)
+ return;
+
+ vc4_hdmi_audio_reset(hdmi);
+
+ hdmi->audio.substream = NULL;
+}
+
+/* HDMI audio codec callbacks */
+static int vc4_hdmi_audio_hw_params(struct snd_pcm_substream *substream,
+ struct snd_pcm_hw_params *params,
+ struct snd_soc_dai *dai)
+{
+ struct vc4_hdmi *hdmi = dai_to_hdmi(dai);
+ struct drm_encoder *encoder = hdmi->encoder;
+ struct drm_device *drm = encoder->dev;
+ struct device *dev = &hdmi->pdev->dev;
+ struct vc4_dev *vc4 = to_vc4_dev(drm);
+ u32 audio_packet_config, channel_mask;
+ u32 channel_map, i;
+
+ if (substream != hdmi->audio.substream)
+ return -EINVAL;
+
+ dev_dbg(dev, "%s: %u Hz, %d bit, %d channels\n", __func__,
+ params_rate(params), params_width(params),
+ params_channels(params));
+
+ hdmi->audio.channels = params_channels(params);
+ hdmi->audio.samplerate = params_rate(params);
+
+ HD_WRITE(VC4_HD_MAI_CTL,
+ VC4_HD_MAI_CTL_RESET |
+ VC4_HD_MAI_CTL_FLUSH |
+ VC4_HD_MAI_CTL_DLATE |
+ VC4_HD_MAI_CTL_ERRORE |
+ VC4_HD_MAI_CTL_ERRORF);
+
+ vc4_hdmi_audio_set_mai_clock(hdmi);
+
+ audio_packet_config =
+ VC4_HDMI_AUDIO_PACKET_ZERO_DATA_ON_SAMPLE_FLAT |
+ VC4_HDMI_AUDIO_PACKET_ZERO_DATA_ON_INACTIVE_CHANNELS |
+ VC4_SET_FIELD(0xf, VC4_HDMI_AUDIO_PACKET_B_FRAME_IDENTIFIER);
+
+ channel_mask = GENMASK(hdmi->audio.channels - 1, 0);
+ audio_packet_config |= VC4_SET_FIELD(channel_mask,
+ VC4_HDMI_AUDIO_PACKET_CEA_MASK);
+
+ /* Set the MAI threshold. This logic mimics the firmware's. */
+ if (hdmi->audio.samplerate > 96000) {
+ HD_WRITE(VC4_HD_MAI_THR,
+ VC4_SET_FIELD(0x12, VC4_HD_MAI_THR_DREQHIGH) |
+ VC4_SET_FIELD(0x12, VC4_HD_MAI_THR_DREQLOW));
+ } else if (hdmi->audio.samplerate > 48000) {
+ HD_WRITE(VC4_HD_MAI_THR,
+ VC4_SET_FIELD(0x14, VC4_HD_MAI_THR_DREQHIGH) |
+ VC4_SET_FIELD(0x12, VC4_HD_MAI_THR_DREQLOW));
+ } else {
+ HD_WRITE(VC4_HD_MAI_THR,
+ VC4_SET_FIELD(0x10, VC4_HD_MAI_THR_PANICHIGH) |
+ VC4_SET_FIELD(0x10, VC4_HD_MAI_THR_PANICLOW) |
+ VC4_SET_FIELD(0x10, VC4_HD_MAI_THR_DREQHIGH) |
+ VC4_SET_FIELD(0x10, VC4_HD_MAI_THR_DREQLOW));
+ }
+
+ HDMI_WRITE(VC4_HDMI_MAI_CONFIG,
+ VC4_HDMI_MAI_CONFIG_BIT_REVERSE |
+ VC4_SET_FIELD(channel_mask, VC4_HDMI_MAI_CHANNEL_MASK));
+
+ channel_map = 0;
+ for (i = 0; i < 8; i++) {
+ if (channel_mask & BIT(i))
+ channel_map |= i << (3 * i);
+ }
+
+ HDMI_WRITE(VC4_HDMI_MAI_CHANNEL_MAP, channel_map);
+ HDMI_WRITE(VC4_HDMI_AUDIO_PACKET_CONFIG, audio_packet_config);
+ vc4_hdmi_set_n_cts(hdmi);
+
+ return 0;
+}
+
+static int vc4_hdmi_audio_trigger(struct snd_pcm_substream *substream, int cmd,
+ struct snd_soc_dai *dai)
+{
+ struct vc4_hdmi *hdmi = dai_to_hdmi(dai);
+ struct drm_encoder *encoder = hdmi->encoder;
+ struct drm_device *drm = encoder->dev;
+ struct vc4_dev *vc4 = to_vc4_dev(drm);
+
+ switch (cmd) {
+ case SNDRV_PCM_TRIGGER_START:
+ vc4_hdmi_set_audio_infoframe(encoder);
+ HDMI_WRITE(VC4_HDMI_TX_PHY_CTL0,
+ HDMI_READ(VC4_HDMI_TX_PHY_CTL0) &
+ ~VC4_HDMI_TX_PHY_RNG_PWRDN);
+ HD_WRITE(VC4_HD_MAI_CTL,
+ VC4_SET_FIELD(hdmi->audio.channels,
+ VC4_HD_MAI_CTL_CHNUM) |
+ VC4_HD_MAI_CTL_ENABLE);
+ break;
+ case SNDRV_PCM_TRIGGER_STOP:
+ HD_WRITE(VC4_HD_MAI_CTL,
+ VC4_HD_MAI_CTL_DLATE |
+ VC4_HD_MAI_CTL_ERRORE |
+ VC4_HD_MAI_CTL_ERRORF);
+ HDMI_WRITE(VC4_HDMI_TX_PHY_CTL0,
+ HDMI_READ(VC4_HDMI_TX_PHY_CTL0) |
+ VC4_HDMI_TX_PHY_RNG_PWRDN);
+ break;
+ default:
+ break;
+ }
+
+ return 0;
+}
+
+static inline struct vc4_hdmi *
+snd_component_to_hdmi(struct snd_soc_component *component)
+{
+ struct snd_soc_card *card = snd_soc_component_get_drvdata(component);
+
+ return snd_soc_card_get_drvdata(card);
+}
+
+static int vc4_hdmi_audio_eld_ctl_info(struct snd_kcontrol *kcontrol,
+ struct snd_ctl_elem_info *uinfo)
+{
+ struct snd_soc_component *component = snd_kcontrol_chip(kcontrol);
+ struct vc4_hdmi *hdmi = snd_component_to_hdmi(component);
+
+ uinfo->type = SNDRV_CTL_ELEM_TYPE_BYTES;
+ uinfo->count = sizeof(hdmi->connector->eld);
+
+ return 0;
+}
+
+static int vc4_hdmi_audio_eld_ctl_get(struct snd_kcontrol *kcontrol,
+ struct snd_ctl_elem_value *ucontrol)
+{
+ struct snd_soc_component *component = snd_kcontrol_chip(kcontrol);
+ struct vc4_hdmi *hdmi = snd_component_to_hdmi(component);
+
+ memcpy(ucontrol->value.bytes.data, hdmi->connector->eld,
+ sizeof(hdmi->connector->eld));
+
+ return 0;
+}
+
+static const struct snd_kcontrol_new vc4_hdmi_audio_controls[] = {
+ {
+ .access = SNDRV_CTL_ELEM_ACCESS_READ |
+ SNDRV_CTL_ELEM_ACCESS_VOLATILE,
+ .iface = SNDRV_CTL_ELEM_IFACE_PCM,
+ .name = "ELD",
+ .info = vc4_hdmi_audio_eld_ctl_info,
+ .get = vc4_hdmi_audio_eld_ctl_get,
+ },
+};
+
+static const struct snd_soc_dapm_widget vc4_hdmi_audio_widgets[] = {
+ SND_SOC_DAPM_OUTPUT("TX"),
+};
+
+static const struct snd_soc_dapm_route vc4_hdmi_audio_routes[] = {
+ { "TX", NULL, "Playback" },
+};
+
+static const struct snd_soc_codec_driver vc4_hdmi_audio_codec_drv = {
+ .component_driver = {
+ .controls = vc4_hdmi_audio_controls,
+ .num_controls = ARRAY_SIZE(vc4_hdmi_audio_controls),
+ .dapm_widgets = vc4_hdmi_audio_widgets,
+ .num_dapm_widgets = ARRAY_SIZE(vc4_hdmi_audio_widgets),
+ .dapm_routes = vc4_hdmi_audio_routes,
+ .num_dapm_routes = ARRAY_SIZE(vc4_hdmi_audio_routes),
+ },
+};
+
+static const struct snd_soc_dai_ops vc4_hdmi_audio_dai_ops = {
+ .startup = vc4_hdmi_audio_startup,
+ .shutdown = vc4_hdmi_audio_shutdown,
+ .hw_params = vc4_hdmi_audio_hw_params,
+ .set_fmt = vc4_hdmi_audio_set_fmt,
+ .trigger = vc4_hdmi_audio_trigger,
+};
+
+static struct snd_soc_dai_driver vc4_hdmi_audio_codec_dai_drv = {
+ .name = "vc4-hdmi-hifi",
+ .playback = {
+ .stream_name = "Playback",
+ .channels_min = 2,
+ .channels_max = 8,
+ .rates = SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |
+ SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_88200 |
+ SNDRV_PCM_RATE_96000 | SNDRV_PCM_RATE_176400 |
+ SNDRV_PCM_RATE_192000,
+ .formats = SNDRV_PCM_FMTBIT_IEC958_SUBFRAME_LE,
+ },
+};
+
+static const struct snd_soc_component_driver vc4_hdmi_audio_cpu_dai_comp = {
+ .name = "vc4-hdmi-cpu-dai-component",
+};
+
+static int vc4_hdmi_audio_cpu_dai_probe(struct snd_soc_dai *dai)
+{
+ struct vc4_hdmi *hdmi = dai_to_hdmi(dai);
+
+ snd_soc_dai_init_dma_data(dai, &hdmi->audio.dma_data, NULL);
+
+ return 0;
+}
+
+static struct snd_soc_dai_driver vc4_hdmi_audio_cpu_dai_drv = {
+ .name = "vc4-hdmi-cpu-dai",
+ .probe = vc4_hdmi_audio_cpu_dai_probe,
+ .playback = {
+ .stream_name = "Playback",
+ .channels_min = 1,
+ .channels_max = 8,
+ .rates = SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |
+ SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_88200 |
+ SNDRV_PCM_RATE_96000 | SNDRV_PCM_RATE_176400 |
+ SNDRV_PCM_RATE_192000,
+ .formats = SNDRV_PCM_FMTBIT_IEC958_SUBFRAME_LE,
+ },
+ .ops = &vc4_hdmi_audio_dai_ops,
+};
+
+static const struct snd_dmaengine_pcm_config pcm_conf = {
+ .chan_names[SNDRV_PCM_STREAM_PLAYBACK] = "audio-rx",
+ .prepare_slave_config = snd_dmaengine_pcm_prepare_slave_config,
+};
+
+static int vc4_hdmi_audio_init(struct vc4_hdmi *hdmi)
+{
+ struct snd_soc_dai_link *dai_link = &hdmi->audio.link;
+ struct snd_soc_card *card = &hdmi->audio.card;
+ struct device *dev = &hdmi->pdev->dev;
+ const __be32 *addr;
+ int ret;
+
+ if (!of_find_property(dev->of_node, "dmas", NULL)) {
+ dev_warn(dev,
+ "'dmas' DT property is missing, no HDMI audio\n");
+ return 0;
+ }
+
+ /*
+ * Get the physical address of VC4_HD_MAI_DATA. We need to retrieve
+ * the bus address specified in the DT, because the physical address
+ * (the one returned by platform_get_resource()) is not appropriate
+ * for DMA transfers.
+ * This VC/MMU should probably be exposed to avoid this kind of hacks.
+ */
+ addr = of_get_address(dev->of_node, 1, NULL, NULL);
+ hdmi->audio.dma_data.addr = be32_to_cpup(addr) + VC4_HD_MAI_DATA;
+ hdmi->audio.dma_data.addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
+ hdmi->audio.dma_data.maxburst = 2;
+
+ ret = devm_snd_dmaengine_pcm_register(dev, &pcm_conf, 0);
+ if (ret) {
+ dev_err(dev, "Could not register PCM component: %d\n", ret);
+ return ret;
+ }
+
+ ret = devm_snd_soc_register_component(dev, &vc4_hdmi_audio_cpu_dai_comp,
+ &vc4_hdmi_audio_cpu_dai_drv, 1);
+ if (ret) {
+ dev_err(dev, "Could not register CPU DAI: %d\n", ret);
+ return ret;
+ }
+
+ /* register codec and codec dai */
+ ret = snd_soc_register_codec(dev, &vc4_hdmi_audio_codec_drv,
+ &vc4_hdmi_audio_codec_dai_drv, 1);
+ if (ret) {
+ dev_err(dev, "Could not register codec: %d\n", ret);
+ return ret;
+ }
+
+ dai_link->name = "MAI";
+ dai_link->stream_name = "MAI PCM";
+ dai_link->codec_dai_name = vc4_hdmi_audio_codec_dai_drv.name;
+ dai_link->cpu_dai_name = dev_name(dev);
+ dai_link->codec_name = dev_name(dev);
+ dai_link->platform_name = dev_name(dev);
+
+ card->dai_link = dai_link;
+ card->num_links = 1;
+ card->name = "vc4-hdmi";
+ card->dev = dev;
+
+ /*
+ * Be careful, snd_soc_register_card() calls dev_set_drvdata() and
+ * stores a pointer to the snd card object in dev->driver_data. This
+ * means we cannot use it for something else. The hdmi back-pointer is
+ * now stored in card->drvdata and should be retrieved with
+ * snd_soc_card_get_drvdata() if needed.
+ */
+ snd_soc_card_set_drvdata(card, hdmi);
+ ret = devm_snd_soc_register_card(dev, card);
+ if (ret) {
+ dev_err(dev, "Could not register sound card: %d\n", ret);
+ goto unregister_codec;
+ }
+
+ return 0;
+
+unregister_codec:
+ snd_soc_unregister_codec(dev);
+
+ return ret;
+}
+
+static void vc4_hdmi_audio_cleanup(struct vc4_hdmi *hdmi)
+{
+ struct device *dev = &hdmi->pdev->dev;
+
+ /*
+ * If drvdata is not set this means the audio card was not
+ * registered, just skip codec unregistration in this case.
+ */
+ if (dev_get_drvdata(dev))
+ snd_soc_unregister_codec(dev);
+}
+
static int vc4_hdmi_bind(struct device *dev, struct device *master, void *data)
{
struct platform_device *pdev = to_platform_device(dev);
goto err_destroy_encoder;
}
+ ret = vc4_hdmi_audio_init(hdmi);
+ if (ret)
+ goto err_destroy_encoder;
+
return 0;
err_destroy_encoder:
struct vc4_dev *vc4 = drm->dev_private;
struct vc4_hdmi *hdmi = vc4->hdmi;
+ vc4_hdmi_audio_cleanup(hdmi);
+
vc4_hdmi_connector_destroy(hdmi->connector);
vc4_hdmi_encoder_destroy(hdmi->encoder);
/**
* DOC: VC4 HVS module.
*
- * The HVS is the piece of hardware that does translation, scaling,
- * colorspace conversion, and compositing of pixels stored in
- * framebuffers into a FIFO of pixels going out to the Pixel Valve
- * (CRTC). It operates at the system clock rate (the system audio
- * clock gate, specifically), which is much higher than the pixel
- * clock rate.
+ * The Hardware Video Scaler (HVS) is the piece of hardware that does
+ * translation, scaling, colorspace conversion, and compositing of
+ * pixels stored in framebuffers into a FIFO of pixels going out to
+ * the Pixel Valve (CRTC). It operates at the system clock rate (the
+ * system audio clock gate, specifically), which is much higher than
+ * the pixel clock rate.
*
* There is a single global HVS, with multiple output FIFOs that can
* be consumed by the PVs. This file just manages the resources for
* IN THE SOFTWARE.
*/
-/** DOC: Interrupt management for the V3D engine.
+/**
+ * DOC: Interrupt management for the V3D engine
*
* We have an interrupt status register (V3D_INTCTL) which reports
* interrupts, and where writing 1 bits clears those interrupts.
#include "vc4_drv.h"
#include "vc4_regs.h"
+#include "drm_atomic.h"
#include "drm_atomic_helper.h"
#include "drm_fb_cma_helper.h"
#include "drm_plane_helper.h"
if (!plane_state)
goto out;
- /* If we're changing the cursor contents, do that in the
- * normal vblank-synced atomic path.
- */
- if (fb != plane_state->fb)
- goto out;
-
/* No configuring new scaling in the fast path. */
if (crtc_w != plane_state->crtc_w ||
crtc_h != plane_state->crtc_h ||
goto out;
}
+ if (fb != plane_state->fb) {
+ drm_atomic_set_fb_for_plane(plane->state, fb);
+ vc4_plane_async_set_fb(plane, fb);
+ }
+
/* Set the cursor's position on the screen. This is the
* expected change from the drm_mode_cursor_universal()
* helper.
vc4_plane = devm_kzalloc(dev->dev, sizeof(*vc4_plane),
GFP_KERNEL);
- if (!vc4_plane) {
- ret = -ENOMEM;
- goto fail;
- }
+ if (!vc4_plane)
+ return ERR_PTR(-ENOMEM);
for (i = 0; i < ARRAY_SIZE(hvs_formats); i++) {
/* Don't allow YUV in cursor planes, since that means
drm_plane_helper_add(plane, &vc4_plane_helper_funcs);
return plane;
-fail:
- if (plane)
- vc4_plane_destroy(plane);
-
- return ERR_PTR(ret);
}
#define VC4_HDMI_HOTPLUG 0x00c
# define VC4_HDMI_HOTPLUG_CONNECTED BIT(0)
+/* 3 bits per field, where each field maps from that corresponding MAI
+ * bus channel to the given HDMI channel.
+ */
+#define VC4_HDMI_MAI_CHANNEL_MAP 0x090
+
+#define VC4_HDMI_MAI_CONFIG 0x094
+# define VC4_HDMI_MAI_CONFIG_FORMAT_REVERSE BIT(27)
+# define VC4_HDMI_MAI_CONFIG_BIT_REVERSE BIT(26)
+# define VC4_HDMI_MAI_CHANNEL_MASK_MASK VC4_MASK(15, 0)
+# define VC4_HDMI_MAI_CHANNEL_MASK_SHIFT 0
+
+/* Last received format word on the MAI bus. */
+#define VC4_HDMI_MAI_FORMAT 0x098
+
+#define VC4_HDMI_AUDIO_PACKET_CONFIG 0x09c
+# define VC4_HDMI_AUDIO_PACKET_ZERO_DATA_ON_SAMPLE_FLAT BIT(29)
+# define VC4_HDMI_AUDIO_PACKET_ZERO_DATA_ON_INACTIVE_CHANNELS BIT(24)
+# define VC4_HDMI_AUDIO_PACKET_FORCE_SAMPLE_PRESENT BIT(19)
+# define VC4_HDMI_AUDIO_PACKET_FORCE_B_FRAME BIT(18)
+# define VC4_HDMI_AUDIO_PACKET_B_FRAME_IDENTIFIER_MASK VC4_MASK(13, 10)
+# define VC4_HDMI_AUDIO_PACKET_B_FRAME_IDENTIFIER_SHIFT 10
+/* If set, then multichannel, otherwise 2 channel. */
+# define VC4_HDMI_AUDIO_PACKET_AUDIO_LAYOUT BIT(9)
+/* If set, then AUDIO_LAYOUT overrides audio_cea_mask */
+# define VC4_HDMI_AUDIO_PACKET_FORCE_AUDIO_LAYOUT BIT(8)
+# define VC4_HDMI_AUDIO_PACKET_CEA_MASK_MASK VC4_MASK(7, 0)
+# define VC4_HDMI_AUDIO_PACKET_CEA_MASK_SHIFT 0
+
#define VC4_HDMI_RAM_PACKET_CONFIG 0x0a0
# define VC4_HDMI_RAM_PACKET_ENABLE BIT(16)
#define VC4_HDMI_RAM_PACKET_STATUS 0x0a4
+#define VC4_HDMI_CRP_CFG 0x0a8
+/* When set, the CTS_PERIOD counts based on MAI bus sync pulse instead
+ * of pixel clock.
+ */
+# define VC4_HDMI_CRP_USE_MAI_BUS_SYNC_FOR_CTS BIT(26)
+/* When set, no CRP packets will be sent. */
+# define VC4_HDMI_CRP_CFG_DISABLE BIT(25)
+/* If set, generates CTS values based on N, audio clock, and video
+ * clock. N must be divisible by 128.
+ */
+# define VC4_HDMI_CRP_CFG_EXTERNAL_CTS_EN BIT(24)
+# define VC4_HDMI_CRP_CFG_N_MASK VC4_MASK(19, 0)
+# define VC4_HDMI_CRP_CFG_N_SHIFT 0
+
+/* 20-bit fields containing CTS values to be transmitted if !EXTERNAL_CTS_EN */
+#define VC4_HDMI_CTS_0 0x0ac
+#define VC4_HDMI_CTS_1 0x0b0
+/* 20-bit fields containing number of clocks to send CTS0/1 before
+ * switching to the other one.
+ */
+#define VC4_HDMI_CTS_PERIOD_0 0x0b4
+#define VC4_HDMI_CTS_PERIOD_1 0x0b8
+
#define VC4_HDMI_HORZA 0x0c4
# define VC4_HDMI_HORZA_VPOS BIT(14)
# define VC4_HDMI_HORZA_HPOS BIT(13)
#define VC4_HDMI_TX_PHY_RESET_CTL 0x2c0
-#define VC4_HDMI_GCP_0 0x400
+#define VC4_HDMI_TX_PHY_CTL0 0x2c4
+# define VC4_HDMI_TX_PHY_RNG_PWRDN BIT(25)
+
+#define VC4_HDMI_GCP(x) (0x400 + ((x) * 0x4))
+#define VC4_HDMI_RAM_PACKET(x) (0x400 + ((x) * 0x24))
#define VC4_HDMI_PACKET_STRIDE 0x24
#define VC4_HD_M_CTL 0x00c
# define VC4_HD_M_ENABLE BIT(0)
#define VC4_HD_MAI_CTL 0x014
+/* Set when audio stream is received at a slower rate than the
+ * sampling period, so MAI fifo goes empty. Write 1 to clear.
+ */
+# define VC4_HD_MAI_CTL_DLATE BIT(15)
+# define VC4_HD_MAI_CTL_BUSY BIT(14)
+# define VC4_HD_MAI_CTL_CHALIGN BIT(13)
+# define VC4_HD_MAI_CTL_WHOLSMP BIT(12)
+# define VC4_HD_MAI_CTL_FULL BIT(11)
+# define VC4_HD_MAI_CTL_EMPTY BIT(10)
+# define VC4_HD_MAI_CTL_FLUSH BIT(9)
+/* If set, MAI bus generates SPDIF (bit 31) parity instead of passing
+ * through.
+ */
+# define VC4_HD_MAI_CTL_PAREN BIT(8)
+# define VC4_HD_MAI_CTL_CHNUM_MASK VC4_MASK(7, 4)
+# define VC4_HD_MAI_CTL_CHNUM_SHIFT 4
+# define VC4_HD_MAI_CTL_ENABLE BIT(3)
+/* Underflow error status bit, write 1 to clear. */
+# define VC4_HD_MAI_CTL_ERRORE BIT(2)
+/* Overflow error status bit, write 1 to clear. */
+# define VC4_HD_MAI_CTL_ERRORF BIT(1)
+/* Single-shot reset bit. Read value is undefined. */
+# define VC4_HD_MAI_CTL_RESET BIT(0)
+
+#define VC4_HD_MAI_THR 0x018
+# define VC4_HD_MAI_THR_PANICHIGH_MASK VC4_MASK(29, 24)
+# define VC4_HD_MAI_THR_PANICHIGH_SHIFT 24
+# define VC4_HD_MAI_THR_PANICLOW_MASK VC4_MASK(21, 16)
+# define VC4_HD_MAI_THR_PANICLOW_SHIFT 16
+# define VC4_HD_MAI_THR_DREQHIGH_MASK VC4_MASK(13, 8)
+# define VC4_HD_MAI_THR_DREQHIGH_SHIFT 8
+# define VC4_HD_MAI_THR_DREQLOW_MASK VC4_MASK(5, 0)
+# define VC4_HD_MAI_THR_DREQLOW_SHIFT 0
+
+/* Format header to be placed on the MAI data. Unused. */
+#define VC4_HD_MAI_FMT 0x01c
+
+/* Register for DMAing in audio data to be transported over the MAI
+ * bus to the Falcon core.
+ */
+#define VC4_HD_MAI_DATA 0x020
+
+/* Divider from HDMI HSM clock to MAI serial clock. Sampling period
+ * converges to N / (M + 1) cycles.
+ */
+#define VC4_HD_MAI_SMP 0x02c
+# define VC4_HD_MAI_SMP_N_MASK VC4_MASK(31, 8)
+# define VC4_HD_MAI_SMP_N_SHIFT 8
+# define VC4_HD_MAI_SMP_M_MASK VC4_MASK(7, 0)
+# define VC4_HD_MAI_SMP_M_SHIFT 0
#define VC4_HD_VID_CTL 0x038
# define VC4_HD_VID_CTL_ENABLE BIT(31)
/**
* DOC: Render command list generation
*
+ * In the V3D hardware, render command lists are what load and store
+ * tiles of a framebuffer and optionally call out to binner-generated
+ * command lists to do the 3D drawing for that tile.
+ *
* In the VC4 driver, render command list generation is performed by the
* kernel instead of userspace. We do this because validating a
* user-submitted command list is hard to get right and has high CPU overhead,
*/
/**
- * Command list validator for VC4.
+ * DOC: Command list validator for VC4.
*
- * The VC4 has no IOMMU between it and system memory. So, a user with
- * access to execute command lists could escalate privilege by
+ * Since the VC4 has no IOMMU between it and system memory, a user
+ * with access to execute command lists could escalate privilege by
* overwriting system memory (drawing to it as a framebuffer) or
- * reading system memory it shouldn't (reading it as a texture, or
- * uniform data, or vertex data).
+ * reading system memory it shouldn't (reading it as a vertex buffer
+ * or index buffer)
*
- * This validates command lists to ensure that all accesses are within
- * the bounds of the GEM objects referenced. It explicitly whitelists
- * packets, and looks at the offsets in any address fields to make
- * sure they're constrained within the BOs they reference.
+ * We validate binner command lists to ensure that all accesses are
+ * within the bounds of the GEM objects referenced by the submitted
+ * job. It explicitly whitelists packets, and looks at the offsets in
+ * any address fields to make sure they're contained within the BOs
+ * they reference.
*
- * Note that because of the validation that's happening anyway, this
- * is where GEM relocation processing happens.
+ * Note that because CL validation is already reading the
+ * user-submitted CL and writing the validated copy out to the memory
+ * that the GPU will actually read, this is also where GEM relocation
+ * processing (turning BO references into actual addresses for the GPU
+ * to use) happens.
*/
#include "uapi/drm/vc4_drm.h"
}
/**
- * The texture unit decides what tiling format a particular miplevel is using
- * this function, so we lay out our miptrees accordingly.
+ * size_is_lt() - Returns whether a miplevel of the given size will
+ * use the lineartile (LT) tiling layout rather than the normal T
+ * tiling layout.
+ * @width: Width in pixels of the miplevel
+ * @height: Height in pixels of the miplevel
+ * @cpp: Bytes per pixel of the pixel format
*/
static bool
size_is_lt(uint32_t width, uint32_t height, int cpp)
/**
* DOC: Shader validator for VC4.
*
- * The VC4 has no IOMMU between it and system memory, so a user with
- * access to execute shaders could escalate privilege by overwriting
- * system memory (using the VPM write address register in the
- * general-purpose DMA mode) or reading system memory it shouldn't
- * (reading it as a texture, or uniform data, or vertex data).
+ * Since the VC4 has no IOMMU between it and system memory, a user
+ * with access to execute shaders could escalate privilege by
+ * overwriting system memory (using the VPM write address register in
+ * the general-purpose DMA mode) or reading system memory it shouldn't
+ * (reading it as a texture, uniform data, or direct-addressed TMU
+ * lookup).
*
- * This walks over a shader BO, ensuring that its accesses are
- * appropriately bounded, and recording how many texture accesses are
- * made and where so that we can do relocations for them in the
+ * The shader validator walks over a shader's BO, ensuring that its
+ * accesses are appropriately bounded, and recording where texture
+ * accesses are made so that we can do relocations for them in the
* uniform stream.
+ *
+ * Shader BO are immutable for their lifetimes (enforced by not
+ * allowing mmaps, GEM prime export, or rendering to from a CL), so
+ * this validation is only performed at BO creation time.
*/
#include "vc4_drv.h"
/**
* DOC: VC4 SDTV module
+ *
+ * The VEC encoder generates PAL or NTSC composite video output.
+ *
+ * TV mode selection is done by an atomic property on the encoder,
+ * because a drm_mode_modeinfo is insufficient to distinguish between
+ * PAL and PAL-M or NTSC and NTSC-J.
*/
#include <drm/drm_atomic_helper.h>
return 0;
}
-static void vgem_preclose(struct drm_device *dev, struct drm_file *file)
+static void vgem_postclose(struct drm_device *dev, struct drm_file *file)
{
struct vgem_file *vfile = file->driver_priv;
static struct drm_driver vgem_driver = {
.driver_features = DRIVER_GEM | DRIVER_PRIME,
.open = vgem_open,
- .preclose = vgem_preclose,
+ .postclose = vgem_postclose,
.gem_free_object_unlocked = vgem_gem_free_object,
.gem_vm_ops = &vgem_gem_vm_ops,
.ioctls = vgem_ioctls,
vsg->pages = vzalloc(sizeof(struct page *) * vsg->num_pages);
if (NULL == vsg->pages)
return -ENOMEM;
- down_read(¤t->mm->mmap_sem);
- ret = get_user_pages((unsigned long)xfer->mem_addr,
- vsg->num_pages,
- (vsg->direction == DMA_FROM_DEVICE) ? FOLL_WRITE : 0,
- vsg->pages, NULL);
-
- up_read(¤t->mm->mmap_sem);
+ ret = get_user_pages_unlocked((unsigned long)xfer->mem_addr,
+ vsg->num_pages, vsg->pages,
+ (vsg->direction == DMA_FROM_DEVICE) ? FOLL_WRITE : 0);
if (ret != vsg->num_pages) {
if (ret < 0)
return ret;
minor->debugfs_root, minor);
return 0;
}
-
-void
-virtio_gpu_debugfs_takedown(struct drm_minor *minor)
-{
- drm_debugfs_remove_files(virtio_gpu_debugfs_list,
- VIRTIO_GPU_DEBUGFS_ENTRIES,
- minor);
-}
drm_atomic_helper_cleanup_planes(dev, state);
}
-static struct drm_mode_config_helper_funcs virtio_mode_config_helpers = {
+static const struct drm_mode_config_helper_funcs virtio_mode_config_helpers = {
.atomic_commit_tail = vgdev_atomic_commit_tail,
};
#if defined(CONFIG_DEBUG_FS)
.debugfs_init = virtio_gpu_debugfs_init,
- .debugfs_cleanup = virtio_gpu_debugfs_takedown,
#endif
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
struct virtio_gpu_queue ctrlq;
struct virtio_gpu_queue cursorq;
- struct list_head free_vbufs;
- spinlock_t free_vbufs_lock;
- void *vbufs;
+ struct kmem_cache *vbufs;
bool vqs_ready;
struct idr resource_idr;
/* virgl debufs */
int virtio_gpu_debugfs_init(struct drm_minor *minor);
-void virtio_gpu_debugfs_takedown(struct drm_minor *minor);
#endif
ret = virtio_gpu_framebuffer_init(dev, &vfbdev->vgfb,
&mode_cmd, &obj->gem_base);
if (ret)
- goto err_fb_init;
+ goto err_fb_alloc;
fb = &vfbdev->vgfb.base;
info->fix.mmio_len = 0;
return 0;
-err_fb_init:
- drm_fb_helper_release_fbi(helper);
err_fb_alloc:
virtio_gpu_cmd_resource_inval_backing(vgdev, resid);
err_obj_attach:
struct virtio_gpu_framebuffer *vgfb = &vgfbdev->vgfb;
drm_fb_helper_unregister_fbi(&vgfbdev->helper);
- drm_fb_helper_release_fbi(&vgfbdev->helper);
if (vgfb->obj)
vgfb->obj = NULL;
static void virtio_gpu_plane_destroy(struct drm_plane *plane)
{
+ drm_plane_cleanup(plane);
kfree(plane);
}
int virtio_gpu_alloc_vbufs(struct virtio_gpu_device *vgdev)
{
- struct virtio_gpu_vbuffer *vbuf;
- int i, size, count = 16;
- void *ptr;
-
- INIT_LIST_HEAD(&vgdev->free_vbufs);
- spin_lock_init(&vgdev->free_vbufs_lock);
- count += virtqueue_get_vring_size(vgdev->ctrlq.vq);
- count += virtqueue_get_vring_size(vgdev->cursorq.vq);
- size = count * VBUFFER_SIZE;
- DRM_INFO("virtio vbuffers: %d bufs, %zdB each, %dkB total.\n",
- count, VBUFFER_SIZE, size / 1024);
-
- vgdev->vbufs = kzalloc(size, GFP_KERNEL);
+ vgdev->vbufs = kmem_cache_create("virtio-gpu-vbufs",
+ VBUFFER_SIZE,
+ __alignof__(struct virtio_gpu_vbuffer),
+ 0, NULL);
if (!vgdev->vbufs)
return -ENOMEM;
-
- for (i = 0, ptr = vgdev->vbufs;
- i < count;
- i++, ptr += VBUFFER_SIZE) {
- vbuf = ptr;
- list_add(&vbuf->list, &vgdev->free_vbufs);
- }
return 0;
}
void virtio_gpu_free_vbufs(struct virtio_gpu_device *vgdev)
{
- struct virtio_gpu_vbuffer *vbuf;
- int i, count = 0;
-
- count += virtqueue_get_vring_size(vgdev->ctrlq.vq);
- count += virtqueue_get_vring_size(vgdev->cursorq.vq);
-
- spin_lock(&vgdev->free_vbufs_lock);
- for (i = 0; i < count; i++) {
- if (WARN_ON(list_empty(&vgdev->free_vbufs))) {
- spin_unlock(&vgdev->free_vbufs_lock);
- return;
- }
- vbuf = list_first_entry(&vgdev->free_vbufs,
- struct virtio_gpu_vbuffer, list);
- list_del(&vbuf->list);
- }
- spin_unlock(&vgdev->free_vbufs_lock);
- kfree(vgdev->vbufs);
+ kmem_cache_destroy(vgdev->vbufs);
+ vgdev->vbufs = NULL;
}
static struct virtio_gpu_vbuffer*
{
struct virtio_gpu_vbuffer *vbuf;
- spin_lock(&vgdev->free_vbufs_lock);
- BUG_ON(list_empty(&vgdev->free_vbufs));
- vbuf = list_first_entry(&vgdev->free_vbufs,
- struct virtio_gpu_vbuffer, list);
- list_del(&vbuf->list);
- spin_unlock(&vgdev->free_vbufs_lock);
+ vbuf = kmem_cache_alloc(vgdev->vbufs, GFP_KERNEL);
+ if (!vbuf)
+ return ERR_PTR(-ENOMEM);
memset(vbuf, 0, VBUFFER_SIZE);
BUG_ON(size > MAX_INLINE_CMD_SIZE);
if (vbuf->resp_size > MAX_INLINE_RESP_SIZE)
kfree(vbuf->resp_buf);
kfree(vbuf->data_buf);
- spin_lock(&vgdev->free_vbufs_lock);
- list_add(&vbuf->list, &vgdev->free_vbufs);
- spin_unlock(&vgdev->free_vbufs_lock);
+ kmem_cache_free(vgdev->vbufs, vbuf);
}
static void reclaim_vbufs(struct virtqueue *vq, struct list_head *reclaim_list)
base = ttm_base_object_lookup(tfile, arg->handle);
if (unlikely(base == NULL)) {
- printk(KERN_ERR "Wait invalid fence object handle "
- "0x%08lx.\n",
+ pr_err("Wait invalid fence object handle 0x%08lx\n",
(unsigned long)arg->handle);
return -EINVAL;
}
base = ttm_base_object_lookup(tfile, arg->handle);
if (unlikely(base == NULL)) {
- printk(KERN_ERR "Fence signaled invalid fence object handle "
- "0x%08lx.\n",
+ pr_err("Fence signaled invalid fence object handle 0x%08lx\n",
(unsigned long)arg->handle);
return -EINVAL;
}
static void vmw_gmrid_man_debug(struct ttm_mem_type_manager *man,
const char *prefix)
{
- printk(KERN_INFO "%s: No debug info available for the GMR "
- "id manager.\n", prefix);
+ pr_info("%s: No debug info available for the GMR id manager\n", prefix);
}
const struct ttm_mem_type_manager_func vmw_gmrid_manager_func = {
base = ttm_base_object_lookup(tfile, handle);
if (unlikely(base == NULL)) {
- printk(KERN_ERR "Invalid buffer object handle 0x%08lx.\n",
+ pr_err("Invalid buffer object handle 0x%08lx\n",
(unsigned long)handle);
return -ESRCH;
}
if (unlikely(ttm_base_object_type(base) != ttm_buffer_type)) {
ttm_base_object_unref(&base);
- printk(KERN_ERR "Invalid buffer object handle 0x%08lx.\n",
+ pr_err("Invalid buffer object handle 0x%08lx\n",
(unsigned long)handle);
return -EINVAL;
}
drm_fbdev_cma_restore_mode(priv->fbdev);
}
-static const struct file_operations zx_drm_fops = {
- .owner = THIS_MODULE,
- .open = drm_open,
- .release = drm_release,
- .unlocked_ioctl = drm_ioctl,
-#ifdef CONFIG_COMPAT
- .compat_ioctl = drm_compat_ioctl,
-#endif
- .poll = drm_poll,
- .read = drm_read,
- .llseek = noop_llseek,
- .mmap = drm_gem_cma_mmap,
-};
+DEFINE_DRM_GEM_CMA_FOPS(zx_drm_fops);
static struct drm_driver zx_drm_driver = {
.driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_PRIME |
DRIVER_ATOMIC,
.lastclose = zx_drm_lastclose,
- .get_vblank_counter = drm_vblank_no_hw_counter,
- .enable_vblank = zx_vou_enable_vblank,
- .disable_vblank = zx_vou_disable_vblank,
.gem_free_object = drm_gem_cma_free_object,
.gem_vm_ops = &drm_gem_cma_vm_ops,
.dumb_create = drm_gem_cma_dumb_create,
.atomic_flush = zx_crtc_atomic_flush,
};
+static int zx_vou_enable_vblank(struct drm_crtc *crtc)
+{
+ struct zx_crtc *zcrtc = to_zx_crtc(crtc);
+ struct zx_vou_hw *vou = crtc_to_vou(crtc);
+ u32 int_frame_mask = zcrtc->bits->int_frame_mask;
+
+ zx_writel_mask(vou->timing + TIMING_INT_CTRL, int_frame_mask,
+ int_frame_mask);
+
+ return 0;
+}
+
+static void zx_vou_disable_vblank(struct drm_crtc *crtc)
+{
+ struct zx_crtc *zcrtc = to_zx_crtc(crtc);
+ struct zx_vou_hw *vou = crtc_to_vou(crtc);
+
+ zx_writel_mask(vou->timing + TIMING_INT_CTRL,
+ zcrtc->bits->int_frame_mask, 0);
+}
+
static const struct drm_crtc_funcs zx_crtc_funcs = {
.destroy = drm_crtc_cleanup,
.set_config = drm_atomic_helper_set_config,
.reset = drm_atomic_helper_crtc_reset,
.atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .enable_vblank = zx_vou_enable_vblank,
+ .disable_vblank = zx_vou_disable_vblank,
};
static int zx_crtc_init(struct drm_device *drm, struct zx_vou_hw *vou,
return 0;
}
-int zx_vou_enable_vblank(struct drm_device *drm, unsigned int pipe)
-{
- struct drm_crtc *crtc;
- struct zx_crtc *zcrtc;
- struct zx_vou_hw *vou;
- u32 int_frame_mask;
-
- crtc = drm_crtc_from_index(drm, pipe);
- if (!crtc)
- return 0;
-
- vou = crtc_to_vou(crtc);
- zcrtc = to_zx_crtc(crtc);
- int_frame_mask = zcrtc->bits->int_frame_mask;
-
- zx_writel_mask(vou->timing + TIMING_INT_CTRL, int_frame_mask,
- int_frame_mask);
-
- return 0;
-}
-
-void zx_vou_disable_vblank(struct drm_device *drm, unsigned int pipe)
-{
- struct drm_crtc *crtc;
- struct zx_crtc *zcrtc;
- struct zx_vou_hw *vou;
-
- crtc = drm_crtc_from_index(drm, pipe);
- if (!crtc)
- return;
-
- vou = crtc_to_vou(crtc);
- zcrtc = to_zx_crtc(crtc);
-
- zx_writel_mask(vou->timing + TIMING_INT_CTRL,
- zcrtc->bits->int_frame_mask, 0);
-}
-
void zx_vou_layer_enable(struct drm_plane *plane)
{
struct zx_crtc *zcrtc = to_zx_crtc(plane->state->crtc);
void zx_vou_config_dividers(struct drm_crtc *crtc,
struct vou_div_config *configs, int num);
-int zx_vou_enable_vblank(struct drm_device *drm, unsigned int pipe);
-void zx_vou_disable_vblank(struct drm_device *drm, unsigned int pipe);
-
void zx_vou_layer_enable(struct drm_plane *plane);
void zx_vou_layer_disable(struct drm_plane *plane);
imx-ipu-v3-objs := ipu-common.o ipu-cpmem.o ipu-csi.o ipu-dc.o ipu-di.o \
ipu-dp.o ipu-dmfc.o ipu-ic.o ipu-image-convert.o \
- ipu-smfc.o ipu-vdi.o
+ ipu-pre.o ipu-prg.o ipu-smfc.o ipu-vdi.o
}
EXPORT_SYMBOL_GPL(ipu_get_num);
-void ipu_srm_dp_sync_update(struct ipu_soc *ipu)
+void ipu_srm_dp_update(struct ipu_soc *ipu, bool sync)
{
u32 val;
val = ipu_cm_read(ipu, IPU_SRM_PRI2);
- val |= 0x8;
+ val &= ~DP_S_SRM_MODE_MASK;
+ val |= sync ? DP_S_SRM_MODE_NEXT_FRAME :
+ DP_S_SRM_MODE_NOW;
ipu_cm_write(ipu, val, IPU_SRM_PRI2);
}
-EXPORT_SYMBOL_GPL(ipu_srm_dp_sync_update);
+EXPORT_SYMBOL_GPL(ipu_srm_dp_update);
enum ipu_color_space ipu_drm_fourcc_to_colorspace(u32 drm_fourcc)
{
case DRM_FORMAT_ABGR8888:
case DRM_FORMAT_RGBA8888:
case DRM_FORMAT_BGRA8888:
+ case DRM_FORMAT_RGB565_A8:
+ case DRM_FORMAT_BGR565_A8:
+ case DRM_FORMAT_RGB888_A8:
+ case DRM_FORMAT_BGR888_A8:
+ case DRM_FORMAT_RGBX8888_A8:
+ case DRM_FORMAT_BGRX8888_A8:
return IPUV3_COLORSPACE_RGB;
case DRM_FORMAT_YUYV:
case DRM_FORMAT_UYVY:
{ .compatible = "fsl,imx51-ipu", .data = &ipu_type_imx51, },
{ .compatible = "fsl,imx53-ipu", .data = &ipu_type_imx53, },
{ .compatible = "fsl,imx6q-ipu", .data = &ipu_type_imx6q, },
+ { .compatible = "fsl,imx6qp-ipu", .data = &ipu_type_imx6q, },
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(of, imx_ipu_dt_ids);
if (!ipu)
return -ENODEV;
+ ipu->id = of_alias_get_id(np, "ipu");
+
+ if (of_device_is_compatible(np, "fsl,imx6qp-ipu")) {
+ ipu->prg_priv = ipu_prg_lookup_by_phandle(&pdev->dev,
+ "fsl,prg", ipu->id);
+ if (!ipu->prg_priv)
+ return -EPROBE_DEFER;
+ }
+
for (i = 0; i < 64; i++)
ipu->channel[i].ipu = ipu;
ipu->devtype = devtype;
ipu->ipu_type = devtype->type;
- ipu->id = of_alias_get_id(np, "ipu");
spin_lock_init(&ipu->lock);
mutex_init(&ipu->channel_lock);
.remove = ipu_remove,
};
-module_platform_driver(imx_ipu_driver);
+static struct platform_driver * const drivers[] = {
+ &ipu_pre_drv,
+ &ipu_prg_drv,
+ &imx_ipu_driver,
+};
+
+static int __init imx_ipu_init(void)
+{
+ return platform_register_drivers(drivers, ARRAY_SIZE(drivers));
+}
+module_init(imx_ipu_init);
+
+static void __exit imx_ipu_exit(void)
+{
+ platform_unregister_drivers(drivers, ARRAY_SIZE(drivers));
+}
+module_exit(imx_ipu_exit);
MODULE_ALIAS("platform:imx-ipuv3");
MODULE_DESCRIPTION("i.MX IPU v3 driver");
#define UV2_OFFSET(pix, x, y) ((pix->width * pix->height) + \
(pix->width * y) + (x))
+#define NUM_ALPHA_CHANNELS 7
+
+/* See Table 37-12. Alpha channels mapping. */
+static int ipu_channel_albm(int ch_num)
+{
+ switch (ch_num) {
+ case IPUV3_CHANNEL_G_MEM_IC_PRP_VF: return 0;
+ case IPUV3_CHANNEL_G_MEM_IC_PP: return 1;
+ case IPUV3_CHANNEL_MEM_FG_SYNC: return 2;
+ case IPUV3_CHANNEL_MEM_FG_ASYNC: return 3;
+ case IPUV3_CHANNEL_MEM_BG_SYNC: return 4;
+ case IPUV3_CHANNEL_MEM_BG_ASYNC: return 5;
+ case IPUV3_CHANNEL_MEM_VDI_PLANE1_COMB: return 6;
+ default:
+ return -EINVAL;
+ }
+}
+
+static void ipu_cpmem_set_separate_alpha(struct ipuv3_channel *ch)
+{
+ struct ipu_soc *ipu = ch->ipu;
+ int albm;
+ u32 val;
+
+ albm = ipu_channel_albm(ch->num);
+ if (albm < 0)
+ return;
+
+ ipu_ch_param_write_field(ch, IPU_FIELD_ALU, 1);
+ ipu_ch_param_write_field(ch, IPU_FIELD_ALBM, albm);
+ ipu_ch_param_write_field(ch, IPU_FIELD_CRE, 1);
+
+ val = ipu_idmac_read(ipu, IDMAC_SEP_ALPHA);
+ val |= BIT(ch->num);
+ ipu_idmac_write(ipu, val, IDMAC_SEP_ALPHA);
+}
+
int ipu_cpmem_set_fmt(struct ipuv3_channel *ch, u32 drm_fourcc)
{
switch (drm_fourcc) {
break;
case DRM_FORMAT_RGBA8888:
case DRM_FORMAT_RGBX8888:
+ case DRM_FORMAT_RGBX8888_A8:
ipu_cpmem_set_format_rgb(ch, &def_rgbx_32);
break;
case DRM_FORMAT_BGRA8888:
case DRM_FORMAT_BGRX8888:
+ case DRM_FORMAT_BGRX8888_A8:
ipu_cpmem_set_format_rgb(ch, &def_bgrx_32);
break;
case DRM_FORMAT_BGR888:
+ case DRM_FORMAT_BGR888_A8:
ipu_cpmem_set_format_rgb(ch, &def_bgr_24);
break;
case DRM_FORMAT_RGB888:
+ case DRM_FORMAT_RGB888_A8:
ipu_cpmem_set_format_rgb(ch, &def_rgb_24);
break;
case DRM_FORMAT_RGB565:
+ case DRM_FORMAT_RGB565_A8:
ipu_cpmem_set_format_rgb(ch, &def_rgb_16);
break;
case DRM_FORMAT_BGR565:
+ case DRM_FORMAT_BGR565_A8:
ipu_cpmem_set_format_rgb(ch, &def_bgr_16);
break;
case DRM_FORMAT_ARGB1555:
return -EINVAL;
}
+ switch (drm_fourcc) {
+ case DRM_FORMAT_RGB565_A8:
+ case DRM_FORMAT_BGR565_A8:
+ case DRM_FORMAT_RGB888_A8:
+ case DRM_FORMAT_BGR888_A8:
+ case DRM_FORMAT_RGBX8888_A8:
+ case DRM_FORMAT_BGRX8888_A8:
+ ipu_ch_param_write_field(ch, IPU_FIELD_WID3, 7);
+ ipu_cpmem_set_separate_alpha(ch);
+ break;
+ default:
+ break;
+ }
+
return 0;
}
EXPORT_SYMBOL_GPL(ipu_cpmem_set_fmt);
{
struct v4l2_pix_format *pix = &image->pix;
int offset, u_offset, v_offset;
+ int ret = 0;
pr_debug("%s: resolution: %dx%d stride: %d\n",
__func__, pix->width, pix->height,
offset = image->rect.left * 3 +
image->rect.top * pix->bytesperline;
break;
+ case V4L2_PIX_FMT_SBGGR8:
+ case V4L2_PIX_FMT_SGBRG8:
+ case V4L2_PIX_FMT_SGRBG8:
+ case V4L2_PIX_FMT_SRGGB8:
+ offset = image->rect.left + image->rect.top * pix->bytesperline;
+ break;
+ case V4L2_PIX_FMT_SBGGR16:
+ case V4L2_PIX_FMT_SGBRG16:
+ case V4L2_PIX_FMT_SGRBG16:
+ case V4L2_PIX_FMT_SRGGB16:
+ offset = image->rect.left * 2 +
+ image->rect.top * pix->bytesperline;
+ break;
default:
- return -EINVAL;
+ /* This should not happen */
+ WARN_ON(1);
+ offset = 0;
+ ret = -EINVAL;
}
ipu_cpmem_set_buffer(ch, 0, image->phys0 + offset);
ipu_cpmem_set_buffer(ch, 1, image->phys1 + offset);
- return 0;
+ return ret;
}
EXPORT_SYMBOL_GPL(ipu_cpmem_set_image);
struct ipu_dc channels[IPU_DC_NUM_CHANNELS];
struct mutex mutex;
struct completion comp;
- int dc_irq;
- int dp_irq;
int use_count;
};
}
EXPORT_SYMBOL_GPL(ipu_dc_enable_channel);
-static irqreturn_t dc_irq_handler(int irq, void *dev_id)
-{
- struct ipu_dc *dc = dev_id;
- u32 reg;
-
- reg = readl(dc->base + DC_WR_CH_CONF);
- reg &= ~DC_WR_CH_CONF_PROG_TYPE_MASK;
- writel(reg, dc->base + DC_WR_CH_CONF);
-
- /* The Freescale BSP kernel clears DIx_COUNTER_RELEASE here */
-
- complete(&dc->priv->comp);
- return IRQ_HANDLED;
-}
-
void ipu_dc_disable_channel(struct ipu_dc *dc)
{
- struct ipu_dc_priv *priv = dc->priv;
- int irq;
- unsigned long ret;
u32 val;
- /* TODO: Handle MEM_FG_SYNC differently from MEM_BG_SYNC */
- if (dc->chno == 1)
- irq = priv->dc_irq;
- else if (dc->chno == 5)
- irq = priv->dp_irq;
- else
- return;
-
- init_completion(&priv->comp);
- enable_irq(irq);
- ret = wait_for_completion_timeout(&priv->comp, msecs_to_jiffies(50));
- disable_irq(irq);
- if (ret == 0) {
- dev_warn(priv->dev, "DC stop timeout after 50 ms\n");
-
- val = readl(dc->base + DC_WR_CH_CONF);
- val &= ~DC_WR_CH_CONF_PROG_TYPE_MASK;
- writel(val, dc->base + DC_WR_CH_CONF);
- }
+ val = readl(dc->base + DC_WR_CH_CONF);
+ val &= ~DC_WR_CH_CONF_PROG_TYPE_MASK;
+ writel(val, dc->base + DC_WR_CH_CONF);
}
EXPORT_SYMBOL_GPL(ipu_dc_disable_channel);
struct ipu_dc_priv *priv;
static int channel_offsets[] = { 0, 0x1c, 0x38, 0x54, 0x58, 0x5c,
0x78, 0, 0x94, 0xb4};
- int i, ret;
+ int i;
priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
if (!priv)
priv->channels[i].base = priv->dc_reg + channel_offsets[i];
}
- priv->dc_irq = ipu_map_irq(ipu, IPU_IRQ_DC_FC_1);
- if (!priv->dc_irq)
- return -EINVAL;
- ret = devm_request_irq(dev, priv->dc_irq, dc_irq_handler, 0, NULL,
- &priv->channels[1]);
- if (ret < 0)
- return ret;
- disable_irq(priv->dc_irq);
- priv->dp_irq = ipu_map_irq(ipu, IPU_IRQ_DP_SF_END);
- if (!priv->dp_irq)
- return -EINVAL;
- ret = devm_request_irq(dev, priv->dp_irq, dc_irq_handler, 0, NULL,
- &priv->channels[5]);
- if (ret < 0)
- return ret;
- disable_irq(priv->dp_irq);
-
writel(DC_WR_CH_CONF_WORD_SIZE_24 | DC_WR_CH_CONF_DISP_ID_PARALLEL(1) |
DC_WR_CH_CONF_PROG_DI_ID,
priv->channels[1].base + DC_WR_CH_CONF);
writel(reg & ~DP_COM_CONF_GWAM, flow->base + DP_COM_CONF);
}
- ipu_srm_dp_sync_update(priv->ipu);
+ ipu_srm_dp_update(priv->ipu, true);
mutex_unlock(&priv->mutex);
writel((x_pos << 16) | y_pos, flow->base + DP_FG_POS);
- ipu_srm_dp_sync_update(priv->ipu);
+ ipu_srm_dp_update(priv->ipu, true);
return 0;
}
flow->out_cs, DP_COM_CONF_CSC_DEF_FG);
}
- ipu_srm_dp_sync_update(priv->ipu);
+ ipu_srm_dp_update(priv->ipu, true);
mutex_unlock(&priv->mutex);
reg |= DP_COM_CONF_FG_EN;
writel(reg, flow->base + DP_COM_CONF);
- ipu_srm_dp_sync_update(priv->ipu);
+ ipu_srm_dp_update(priv->ipu, true);
mutex_unlock(&priv->mutex);
}
EXPORT_SYMBOL_GPL(ipu_dp_enable_channel);
-void ipu_dp_disable_channel(struct ipu_dp *dp)
+void ipu_dp_disable_channel(struct ipu_dp *dp, bool sync)
{
struct ipu_flow *flow = to_flow(dp);
struct ipu_dp_priv *priv = flow->priv;
writel(reg, flow->base + DP_COM_CONF);
writel(0, flow->base + DP_FG_POS);
- ipu_srm_dp_sync_update(priv->ipu);
-
- if (ipu_idmac_channel_busy(priv->ipu, IPUV3_CHANNEL_MEM_BG_SYNC))
- ipu_wait_interrupt(priv->ipu, IPU_IRQ_DP_SF_END, 50);
+ ipu_srm_dp_update(priv->ipu, sync);
mutex_unlock(&priv->mutex);
}
ipu_ic_task_idma_init(chan->ic, channel, width, height,
burst_size, rot_mode);
- ipu_cpmem_set_axi_id(channel, 1);
+ /*
+ * Setting a non-zero AXI ID collides with the PRG AXI snooping, so
+ * only do this when there is no PRG present.
+ */
+ if (!channel->ipu->prg_priv)
+ ipu_cpmem_set_axi_id(channel, 1);
ipu_idmac_set_double_buffer(channel, ctx->double_buffering);
}
--- /dev/null
+/*
+ * Copyright (c) 2017 Lucas Stach, Pengutronix
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+#include <drm/drm_fourcc.h>
+#include <linux/clk.h>
+#include <linux/err.h>
+#include <linux/genalloc.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <video/imx-ipu-v3.h>
+
+#include "ipu-prv.h"
+
+#define IPU_PRE_MAX_WIDTH 2048
+#define IPU_PRE_NUM_SCANLINES 8
+
+#define IPU_PRE_CTRL 0x000
+#define IPU_PRE_CTRL_SET 0x004
+#define IPU_PRE_CTRL_ENABLE (1 << 0)
+#define IPU_PRE_CTRL_BLOCK_EN (1 << 1)
+#define IPU_PRE_CTRL_BLOCK_16 (1 << 2)
+#define IPU_PRE_CTRL_SDW_UPDATE (1 << 4)
+#define IPU_PRE_CTRL_VFLIP (1 << 5)
+#define IPU_PRE_CTRL_SO (1 << 6)
+#define IPU_PRE_CTRL_INTERLACED_FIELD (1 << 7)
+#define IPU_PRE_CTRL_HANDSHAKE_EN (1 << 8)
+#define IPU_PRE_CTRL_HANDSHAKE_LINE_NUM(v) ((v & 0x3) << 9)
+#define IPU_PRE_CTRL_HANDSHAKE_ABORT_SKIP_EN (1 << 11)
+#define IPU_PRE_CTRL_EN_REPEAT (1 << 28)
+#define IPU_PRE_CTRL_TPR_REST_SEL (1 << 29)
+#define IPU_PRE_CTRL_CLKGATE (1 << 30)
+#define IPU_PRE_CTRL_SFTRST (1 << 31)
+
+#define IPU_PRE_CUR_BUF 0x030
+
+#define IPU_PRE_NEXT_BUF 0x040
+
+#define IPU_PRE_TPR_CTRL 0x070
+#define IPU_PRE_TPR_CTRL_TILE_FORMAT(v) ((v & 0xff) << 0)
+#define IPU_PRE_TPR_CTRL_TILE_FORMAT_MASK 0xff
+
+#define IPU_PRE_PREFETCH_ENG_CTRL 0x080
+#define IPU_PRE_PREF_ENG_CTRL_PREFETCH_EN (1 << 0)
+#define IPU_PRE_PREF_ENG_CTRL_RD_NUM_BYTES(v) ((v & 0x7) << 1)
+#define IPU_PRE_PREF_ENG_CTRL_INPUT_ACTIVE_BPP(v) ((v & 0x3) << 4)
+#define IPU_PRE_PREF_ENG_CTRL_INPUT_PIXEL_FORMAT(v) ((v & 0x7) << 8)
+#define IPU_PRE_PREF_ENG_CTRL_SHIFT_BYPASS (1 << 11)
+#define IPU_PRE_PREF_ENG_CTRL_FIELD_INVERSE (1 << 12)
+#define IPU_PRE_PREF_ENG_CTRL_PARTIAL_UV_SWAP (1 << 14)
+#define IPU_PRE_PREF_ENG_CTRL_TPR_COOR_OFFSET_EN (1 << 15)
+
+#define IPU_PRE_PREFETCH_ENG_INPUT_SIZE 0x0a0
+#define IPU_PRE_PREFETCH_ENG_INPUT_SIZE_WIDTH(v) ((v & 0xffff) << 0)
+#define IPU_PRE_PREFETCH_ENG_INPUT_SIZE_HEIGHT(v) ((v & 0xffff) << 16)
+
+#define IPU_PRE_PREFETCH_ENG_PITCH 0x0d0
+#define IPU_PRE_PREFETCH_ENG_PITCH_Y(v) ((v & 0xffff) << 0)
+#define IPU_PRE_PREFETCH_ENG_PITCH_UV(v) ((v & 0xffff) << 16)
+
+#define IPU_PRE_STORE_ENG_CTRL 0x110
+#define IPU_PRE_STORE_ENG_CTRL_STORE_EN (1 << 0)
+#define IPU_PRE_STORE_ENG_CTRL_WR_NUM_BYTES(v) ((v & 0x7) << 1)
+#define IPU_PRE_STORE_ENG_CTRL_OUTPUT_ACTIVE_BPP(v) ((v & 0x3) << 4)
+
+#define IPU_PRE_STORE_ENG_SIZE 0x130
+#define IPU_PRE_STORE_ENG_SIZE_INPUT_WIDTH(v) ((v & 0xffff) << 0)
+#define IPU_PRE_STORE_ENG_SIZE_INPUT_HEIGHT(v) ((v & 0xffff) << 16)
+
+#define IPU_PRE_STORE_ENG_PITCH 0x140
+#define IPU_PRE_STORE_ENG_PITCH_OUT_PITCH(v) ((v & 0xffff) << 0)
+
+#define IPU_PRE_STORE_ENG_ADDR 0x150
+
+struct ipu_pre {
+ struct list_head list;
+ struct device *dev;
+
+ void __iomem *regs;
+ struct clk *clk_axi;
+ struct gen_pool *iram;
+
+ dma_addr_t buffer_paddr;
+ void *buffer_virt;
+ bool in_use;
+};
+
+static DEFINE_MUTEX(ipu_pre_list_mutex);
+static LIST_HEAD(ipu_pre_list);
+static int available_pres;
+
+int ipu_pre_get_available_count(void)
+{
+ return available_pres;
+}
+
+struct ipu_pre *
+ipu_pre_lookup_by_phandle(struct device *dev, const char *name, int index)
+{
+ struct device_node *pre_node = of_parse_phandle(dev->of_node,
+ name, index);
+ struct ipu_pre *pre;
+
+ mutex_lock(&ipu_pre_list_mutex);
+ list_for_each_entry(pre, &ipu_pre_list, list) {
+ if (pre_node == pre->dev->of_node) {
+ mutex_unlock(&ipu_pre_list_mutex);
+ device_link_add(dev, pre->dev, DL_FLAG_AUTOREMOVE);
+ return pre;
+ }
+ }
+ mutex_unlock(&ipu_pre_list_mutex);
+
+ return NULL;
+}
+
+int ipu_pre_get(struct ipu_pre *pre)
+{
+ u32 val;
+
+ if (pre->in_use)
+ return -EBUSY;
+
+ clk_prepare_enable(pre->clk_axi);
+
+ /* first get the engine out of reset and remove clock gating */
+ writel(0, pre->regs + IPU_PRE_CTRL);
+
+ /* init defaults that should be applied to all streams */
+ val = IPU_PRE_CTRL_HANDSHAKE_ABORT_SKIP_EN |
+ IPU_PRE_CTRL_HANDSHAKE_EN |
+ IPU_PRE_CTRL_TPR_REST_SEL |
+ IPU_PRE_CTRL_BLOCK_16 | IPU_PRE_CTRL_SDW_UPDATE;
+ writel(val, pre->regs + IPU_PRE_CTRL);
+
+ pre->in_use = true;
+ return 0;
+}
+
+void ipu_pre_put(struct ipu_pre *pre)
+{
+ u32 val;
+
+ val = IPU_PRE_CTRL_SFTRST | IPU_PRE_CTRL_CLKGATE;
+ writel(val, pre->regs + IPU_PRE_CTRL);
+
+ clk_disable_unprepare(pre->clk_axi);
+
+ pre->in_use = false;
+}
+
+void ipu_pre_configure(struct ipu_pre *pre, unsigned int width,
+ unsigned int height, unsigned int stride, u32 format,
+ unsigned int bufaddr)
+{
+ const struct drm_format_info *info = drm_format_info(format);
+ u32 active_bpp = info->cpp[0] >> 1;
+ u32 val;
+
+ writel(bufaddr, pre->regs + IPU_PRE_CUR_BUF);
+ writel(bufaddr, pre->regs + IPU_PRE_NEXT_BUF);
+
+ val = IPU_PRE_PREF_ENG_CTRL_INPUT_PIXEL_FORMAT(0) |
+ IPU_PRE_PREF_ENG_CTRL_INPUT_ACTIVE_BPP(active_bpp) |
+ IPU_PRE_PREF_ENG_CTRL_RD_NUM_BYTES(4) |
+ IPU_PRE_PREF_ENG_CTRL_SHIFT_BYPASS |
+ IPU_PRE_PREF_ENG_CTRL_PREFETCH_EN;
+ writel(val, pre->regs + IPU_PRE_PREFETCH_ENG_CTRL);
+
+ val = IPU_PRE_PREFETCH_ENG_INPUT_SIZE_WIDTH(width) |
+ IPU_PRE_PREFETCH_ENG_INPUT_SIZE_HEIGHT(height);
+ writel(val, pre->regs + IPU_PRE_PREFETCH_ENG_INPUT_SIZE);
+
+ val = IPU_PRE_PREFETCH_ENG_PITCH_Y(stride);
+ writel(val, pre->regs + IPU_PRE_PREFETCH_ENG_PITCH);
+
+ val = IPU_PRE_STORE_ENG_CTRL_OUTPUT_ACTIVE_BPP(active_bpp) |
+ IPU_PRE_STORE_ENG_CTRL_WR_NUM_BYTES(4) |
+ IPU_PRE_STORE_ENG_CTRL_STORE_EN;
+ writel(val, pre->regs + IPU_PRE_STORE_ENG_CTRL);
+
+ val = IPU_PRE_STORE_ENG_SIZE_INPUT_WIDTH(width) |
+ IPU_PRE_STORE_ENG_SIZE_INPUT_HEIGHT(height);
+ writel(val, pre->regs + IPU_PRE_STORE_ENG_SIZE);
+
+ val = IPU_PRE_STORE_ENG_PITCH_OUT_PITCH(stride);
+ writel(val, pre->regs + IPU_PRE_STORE_ENG_PITCH);
+
+ writel(pre->buffer_paddr, pre->regs + IPU_PRE_STORE_ENG_ADDR);
+
+ val = readl(pre->regs + IPU_PRE_CTRL);
+ val |= IPU_PRE_CTRL_EN_REPEAT | IPU_PRE_CTRL_ENABLE |
+ IPU_PRE_CTRL_SDW_UPDATE;
+ writel(val, pre->regs + IPU_PRE_CTRL);
+}
+
+void ipu_pre_update(struct ipu_pre *pre, unsigned int bufaddr)
+{
+ writel(bufaddr, pre->regs + IPU_PRE_NEXT_BUF);
+ writel(IPU_PRE_CTRL_SDW_UPDATE, pre->regs + IPU_PRE_CTRL_SET);
+}
+
+u32 ipu_pre_get_baddr(struct ipu_pre *pre)
+{
+ return (u32)pre->buffer_paddr;
+}
+
+static int ipu_pre_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct resource *res;
+ struct ipu_pre *pre;
+
+ pre = devm_kzalloc(dev, sizeof(*pre), GFP_KERNEL);
+ if (!pre)
+ return -ENOMEM;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ pre->regs = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(pre->regs))
+ return PTR_ERR(pre->regs);
+
+ pre->clk_axi = devm_clk_get(dev, "axi");
+ if (IS_ERR(pre->clk_axi))
+ return PTR_ERR(pre->clk_axi);
+
+ pre->iram = of_gen_pool_get(dev->of_node, "fsl,iram", 0);
+ if (!pre->iram)
+ return -EPROBE_DEFER;
+
+ /*
+ * Allocate IRAM buffer with maximum size. This could be made dynamic,
+ * but as there is no other user of this IRAM region and we can fit all
+ * max sized buffers into it, there is no need yet.
+ */
+ pre->buffer_virt = gen_pool_dma_alloc(pre->iram, IPU_PRE_MAX_WIDTH *
+ IPU_PRE_NUM_SCANLINES * 4,
+ &pre->buffer_paddr);
+ if (!pre->buffer_virt)
+ return -ENOMEM;
+
+ pre->dev = dev;
+ platform_set_drvdata(pdev, pre);
+ mutex_lock(&ipu_pre_list_mutex);
+ list_add(&pre->list, &ipu_pre_list);
+ available_pres++;
+ mutex_unlock(&ipu_pre_list_mutex);
+
+ return 0;
+}
+
+static int ipu_pre_remove(struct platform_device *pdev)
+{
+ struct ipu_pre *pre = platform_get_drvdata(pdev);
+
+ mutex_lock(&ipu_pre_list_mutex);
+ list_del(&pre->list);
+ available_pres--;
+ mutex_unlock(&ipu_pre_list_mutex);
+
+ if (pre->buffer_virt)
+ gen_pool_free(pre->iram, (unsigned long)pre->buffer_virt,
+ IPU_PRE_MAX_WIDTH * IPU_PRE_NUM_SCANLINES * 4);
+ return 0;
+}
+
+static const struct of_device_id ipu_pre_dt_ids[] = {
+ { .compatible = "fsl,imx6qp-pre", },
+ { /* sentinel */ },
+};
+
+struct platform_driver ipu_pre_drv = {
+ .probe = ipu_pre_probe,
+ .remove = ipu_pre_remove,
+ .driver = {
+ .name = "imx-ipu-pre",
+ .of_match_table = ipu_pre_dt_ids,
+ },
+};
--- /dev/null
+/*
+ * Copyright (c) 2016-2017 Lucas Stach, Pengutronix
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+#include <drm/drm_fourcc.h>
+#include <linux/clk.h>
+#include <linux/err.h>
+#include <linux/mfd/syscon.h>
+#include <linux/mfd/syscon/imx6q-iomuxc-gpr.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <video/imx-ipu-v3.h>
+
+#include "ipu-prv.h"
+
+#define IPU_PRG_CTL 0x00
+#define IPU_PRG_CTL_BYPASS(i) (1 << (0 + i))
+#define IPU_PRG_CTL_SOFT_ARID_MASK 0x3
+#define IPU_PRG_CTL_SOFT_ARID_SHIFT(i) (8 + i * 2)
+#define IPU_PRG_CTL_SOFT_ARID(i, v) ((v & 0x3) << (8 + 2 * i))
+#define IPU_PRG_CTL_SO(i) (1 << (16 + i))
+#define IPU_PRG_CTL_VFLIP(i) (1 << (19 + i))
+#define IPU_PRG_CTL_BLOCK_MODE(i) (1 << (22 + i))
+#define IPU_PRG_CTL_CNT_LOAD_EN(i) (1 << (25 + i))
+#define IPU_PRG_CTL_SOFTRST (1 << 30)
+#define IPU_PRG_CTL_SHADOW_EN (1 << 31)
+
+#define IPU_PRG_STATUS 0x04
+#define IPU_PRG_STATUS_BUFFER0_READY(i) (1 << (0 + i * 2))
+#define IPU_PRG_STATUS_BUFFER1_READY(i) (1 << (1 + i * 2))
+
+#define IPU_PRG_QOS 0x08
+#define IPU_PRG_QOS_ARID_MASK 0xf
+#define IPU_PRG_QOS_ARID_SHIFT(i) (0 + i * 4)
+
+#define IPU_PRG_REG_UPDATE 0x0c
+#define IPU_PRG_REG_UPDATE_REG_UPDATE (1 << 0)
+
+#define IPU_PRG_STRIDE(i) (0x10 + i * 0x4)
+#define IPU_PRG_STRIDE_STRIDE_MASK 0x3fff
+
+#define IPU_PRG_CROP_LINE 0x1c
+
+#define IPU_PRG_THD 0x20
+
+#define IPU_PRG_BADDR(i) (0x24 + i * 0x4)
+
+#define IPU_PRG_OFFSET(i) (0x30 + i * 0x4)
+
+#define IPU_PRG_ILO(i) (0x3c + i * 0x4)
+
+#define IPU_PRG_HEIGHT(i) (0x48 + i * 0x4)
+#define IPU_PRG_HEIGHT_PRE_HEIGHT_MASK 0xfff
+#define IPU_PRG_HEIGHT_PRE_HEIGHT_SHIFT 0
+#define IPU_PRG_HEIGHT_IPU_HEIGHT_MASK 0xfff
+#define IPU_PRG_HEIGHT_IPU_HEIGHT_SHIFT 16
+
+struct ipu_prg_channel {
+ bool enabled;
+ int used_pre;
+};
+
+struct ipu_prg {
+ struct list_head list;
+ struct device *dev;
+ int id;
+
+ void __iomem *regs;
+ struct clk *clk_ipg, *clk_axi;
+ struct regmap *iomuxc_gpr;
+ struct ipu_pre *pres[3];
+
+ struct ipu_prg_channel chan[3];
+};
+
+static DEFINE_MUTEX(ipu_prg_list_mutex);
+static LIST_HEAD(ipu_prg_list);
+
+struct ipu_prg *
+ipu_prg_lookup_by_phandle(struct device *dev, const char *name, int ipu_id)
+{
+ struct device_node *prg_node = of_parse_phandle(dev->of_node,
+ name, 0);
+ struct ipu_prg *prg;
+
+ mutex_lock(&ipu_prg_list_mutex);
+ list_for_each_entry(prg, &ipu_prg_list, list) {
+ if (prg_node == prg->dev->of_node) {
+ mutex_unlock(&ipu_prg_list_mutex);
+ device_link_add(dev, prg->dev, DL_FLAG_AUTOREMOVE);
+ prg->id = ipu_id;
+ return prg;
+ }
+ }
+ mutex_unlock(&ipu_prg_list_mutex);
+
+ return NULL;
+}
+
+int ipu_prg_max_active_channels(void)
+{
+ return ipu_pre_get_available_count();
+}
+EXPORT_SYMBOL_GPL(ipu_prg_max_active_channels);
+
+bool ipu_prg_present(struct ipu_soc *ipu)
+{
+ if (ipu->prg_priv)
+ return true;
+
+ return false;
+}
+EXPORT_SYMBOL_GPL(ipu_prg_present);
+
+bool ipu_prg_format_supported(struct ipu_soc *ipu, uint32_t format,
+ uint64_t modifier)
+{
+ const struct drm_format_info *info = drm_format_info(format);
+
+ if (info->num_planes != 1)
+ return false;
+
+ return true;
+}
+EXPORT_SYMBOL_GPL(ipu_prg_format_supported);
+
+int ipu_prg_enable(struct ipu_soc *ipu)
+{
+ struct ipu_prg *prg = ipu->prg_priv;
+ int ret;
+
+ if (!prg)
+ return 0;
+
+ ret = clk_prepare_enable(prg->clk_axi);
+ if (ret)
+ goto fail_disable_ipg;
+
+ return 0;
+
+fail_disable_ipg:
+ clk_disable_unprepare(prg->clk_ipg);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(ipu_prg_enable);
+
+void ipu_prg_disable(struct ipu_soc *ipu)
+{
+ struct ipu_prg *prg = ipu->prg_priv;
+
+ if (!prg)
+ return;
+
+ clk_disable_unprepare(prg->clk_axi);
+}
+EXPORT_SYMBOL_GPL(ipu_prg_disable);
+
+/*
+ * The channel configuartion functions below are not thread safe, as they
+ * must be only called from the atomic commit path in the DRM driver, which
+ * is properly serialized.
+ */
+static int ipu_prg_ipu_to_prg_chan(int ipu_chan)
+{
+ /*
+ * This isn't clearly documented in the RM, but IPU to PRG channel
+ * assignment is fixed, as only with this mapping the control signals
+ * match up.
+ */
+ switch (ipu_chan) {
+ case IPUV3_CHANNEL_MEM_BG_SYNC:
+ return 0;
+ case IPUV3_CHANNEL_MEM_FG_SYNC:
+ return 1;
+ case IPUV3_CHANNEL_MEM_DC_SYNC:
+ return 2;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int ipu_prg_get_pre(struct ipu_prg *prg, int prg_chan)
+{
+ int i, ret;
+
+ /* channel 0 is special as it is hardwired to one of the PREs */
+ if (prg_chan == 0) {
+ ret = ipu_pre_get(prg->pres[0]);
+ if (ret)
+ goto fail;
+ prg->chan[prg_chan].used_pre = 0;
+ return 0;
+ }
+
+ for (i = 1; i < 3; i++) {
+ ret = ipu_pre_get(prg->pres[i]);
+ if (!ret) {
+ u32 val, mux;
+ int shift;
+
+ prg->chan[prg_chan].used_pre = i;
+
+ /* configure the PRE to PRG channel mux */
+ shift = (i == 1) ? 12 : 14;
+ mux = (prg->id << 1) | (prg_chan - 1);
+ regmap_update_bits(prg->iomuxc_gpr, IOMUXC_GPR5,
+ 0x3 << shift, mux << shift);
+
+ /* check other mux, must not point to same channel */
+ shift = (i == 1) ? 14 : 12;
+ regmap_read(prg->iomuxc_gpr, IOMUXC_GPR5, &val);
+ if (((val >> shift) & 0x3) == mux) {
+ regmap_update_bits(prg->iomuxc_gpr, IOMUXC_GPR5,
+ 0x3 << shift,
+ (mux ^ 0x1) << shift);
+ }
+
+ return 0;
+ }
+ }
+
+fail:
+ dev_err(prg->dev, "could not get PRE for PRG chan %d", prg_chan);
+ return ret;
+}
+
+static void ipu_prg_put_pre(struct ipu_prg *prg, int prg_chan)
+{
+ struct ipu_prg_channel *chan = &prg->chan[prg_chan];
+
+ ipu_pre_put(prg->pres[chan->used_pre]);
+ chan->used_pre = -1;
+}
+
+void ipu_prg_channel_disable(struct ipuv3_channel *ipu_chan)
+{
+ int prg_chan = ipu_prg_ipu_to_prg_chan(ipu_chan->num);
+ struct ipu_prg *prg = ipu_chan->ipu->prg_priv;
+ struct ipu_prg_channel *chan = &prg->chan[prg_chan];
+ u32 val;
+
+ if (!chan->enabled || prg_chan < 0)
+ return;
+
+ clk_prepare_enable(prg->clk_ipg);
+
+ val = readl(prg->regs + IPU_PRG_CTL);
+ val |= IPU_PRG_CTL_BYPASS(prg_chan);
+ writel(val, prg->regs + IPU_PRG_CTL);
+
+ val = IPU_PRG_REG_UPDATE_REG_UPDATE;
+ writel(val, prg->regs + IPU_PRG_REG_UPDATE);
+
+ clk_disable_unprepare(prg->clk_ipg);
+
+ ipu_prg_put_pre(prg, prg_chan);
+
+ chan->enabled = false;
+}
+EXPORT_SYMBOL_GPL(ipu_prg_channel_disable);
+
+int ipu_prg_channel_configure(struct ipuv3_channel *ipu_chan,
+ unsigned int axi_id, unsigned int width,
+ unsigned int height, unsigned int stride,
+ u32 format, unsigned long *eba)
+{
+ int prg_chan = ipu_prg_ipu_to_prg_chan(ipu_chan->num);
+ struct ipu_prg *prg = ipu_chan->ipu->prg_priv;
+ struct ipu_prg_channel *chan = &prg->chan[prg_chan];
+ u32 val;
+ int ret;
+
+ if (prg_chan < 0)
+ return prg_chan;
+
+ if (chan->enabled) {
+ ipu_pre_update(prg->pres[chan->used_pre], *eba);
+ return 0;
+ }
+
+ ret = ipu_prg_get_pre(prg, prg_chan);
+ if (ret)
+ return ret;
+
+ ipu_pre_configure(prg->pres[chan->used_pre],
+ width, height, stride, format, *eba);
+
+
+ ret = clk_prepare_enable(prg->clk_ipg);
+ if (ret) {
+ ipu_prg_put_pre(prg, prg_chan);
+ return ret;
+ }
+
+ val = (stride - 1) & IPU_PRG_STRIDE_STRIDE_MASK;
+ writel(val, prg->regs + IPU_PRG_STRIDE(prg_chan));
+
+ val = ((height & IPU_PRG_HEIGHT_PRE_HEIGHT_MASK) <<
+ IPU_PRG_HEIGHT_PRE_HEIGHT_SHIFT) |
+ ((height & IPU_PRG_HEIGHT_IPU_HEIGHT_MASK) <<
+ IPU_PRG_HEIGHT_IPU_HEIGHT_SHIFT);
+ writel(val, prg->regs + IPU_PRG_HEIGHT(prg_chan));
+
+ val = ipu_pre_get_baddr(prg->pres[chan->used_pre]);
+ *eba = val;
+ writel(val, prg->regs + IPU_PRG_BADDR(prg_chan));
+
+ val = readl(prg->regs + IPU_PRG_CTL);
+ /* counter load enable */
+ val |= IPU_PRG_CTL_CNT_LOAD_EN(prg_chan);
+ /* config AXI ID */
+ val &= ~(IPU_PRG_CTL_SOFT_ARID_MASK <<
+ IPU_PRG_CTL_SOFT_ARID_SHIFT(prg_chan));
+ val |= IPU_PRG_CTL_SOFT_ARID(prg_chan, axi_id);
+ /* enable channel */
+ val &= ~IPU_PRG_CTL_BYPASS(prg_chan);
+ writel(val, prg->regs + IPU_PRG_CTL);
+
+ val = IPU_PRG_REG_UPDATE_REG_UPDATE;
+ writel(val, prg->regs + IPU_PRG_REG_UPDATE);
+
+ clk_disable_unprepare(prg->clk_ipg);
+
+ chan->enabled = true;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_prg_channel_configure);
+
+static int ipu_prg_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct resource *res;
+ struct ipu_prg *prg;
+ u32 val;
+ int i, ret;
+
+ prg = devm_kzalloc(dev, sizeof(*prg), GFP_KERNEL);
+ if (!prg)
+ return -ENOMEM;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ prg->regs = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(prg->regs))
+ return PTR_ERR(prg->regs);
+
+
+ prg->clk_ipg = devm_clk_get(dev, "ipg");
+ if (IS_ERR(prg->clk_ipg))
+ return PTR_ERR(prg->clk_ipg);
+
+ prg->clk_axi = devm_clk_get(dev, "axi");
+ if (IS_ERR(prg->clk_axi))
+ return PTR_ERR(prg->clk_axi);
+
+ prg->iomuxc_gpr =
+ syscon_regmap_lookup_by_compatible("fsl,imx6q-iomuxc-gpr");
+ if (IS_ERR(prg->iomuxc_gpr))
+ return PTR_ERR(prg->iomuxc_gpr);
+
+ for (i = 0; i < 3; i++) {
+ prg->pres[i] = ipu_pre_lookup_by_phandle(dev, "fsl,pres", i);
+ if (!prg->pres[i])
+ return -EPROBE_DEFER;
+ }
+
+ ret = clk_prepare_enable(prg->clk_ipg);
+ if (ret)
+ return ret;
+
+ /* init to free running mode */
+ val = readl(prg->regs + IPU_PRG_CTL);
+ val |= IPU_PRG_CTL_SHADOW_EN;
+ writel(val, prg->regs + IPU_PRG_CTL);
+
+ /* disable address threshold */
+ writel(0xffffffff, prg->regs + IPU_PRG_THD);
+
+ clk_disable_unprepare(prg->clk_ipg);
+
+ prg->dev = dev;
+ platform_set_drvdata(pdev, prg);
+ mutex_lock(&ipu_prg_list_mutex);
+ list_add(&prg->list, &ipu_prg_list);
+ mutex_unlock(&ipu_prg_list_mutex);
+
+ return 0;
+}
+
+static int ipu_prg_remove(struct platform_device *pdev)
+{
+ struct ipu_prg *prg = platform_get_drvdata(pdev);
+
+ mutex_lock(&ipu_prg_list_mutex);
+ list_del(&prg->list);
+ mutex_unlock(&ipu_prg_list_mutex);
+
+ return 0;
+}
+
+static const struct of_device_id ipu_prg_dt_ids[] = {
+ { .compatible = "fsl,imx6qp-prg", },
+ { /* sentinel */ },
+};
+
+struct platform_driver ipu_prg_drv = {
+ .probe = ipu_prg_probe,
+ .remove = ipu_prg_remove,
+ .driver = {
+ .name = "imx-ipu-prg",
+ .of_match_table = ipu_prg_dt_ids,
+ },
+};
#define IPU_INT_CTRL(n) IPU_CM_REG(0x003C + 4 * (n))
#define IPU_INT_STAT(n) IPU_CM_REG(0x0200 + 4 * (n))
+/* SRM_PRI2 */
+#define DP_S_SRM_MODE_MASK (0x3 << 3)
+#define DP_S_SRM_MODE_NOW (0x3 << 3)
+#define DP_S_SRM_MODE_NEXT_FRAME (0x1 << 3)
+
/* FS_PROC_FLOW1 */
#define FS_PRPENC_ROT_SRC_SEL_MASK (0xf << 0)
#define FS_PRPENC_ROT_SRC_SEL_ENC (0x7 << 0)
struct ipu_vdi;
struct ipu_image_convert_priv;
struct ipu_smfc_priv;
+struct ipu_pre;
+struct ipu_prg;
struct ipu_devtype;
struct ipu_vdi *vdi_priv;
struct ipu_image_convert_priv *image_convert_priv;
struct ipu_smfc_priv *smfc_priv;
+ struct ipu_prg *prg_priv;
};
static inline u32 ipu_idmac_read(struct ipu_soc *ipu, unsigned offset)
writel(value, ipu->idmac_reg + offset);
}
-void ipu_srm_dp_sync_update(struct ipu_soc *ipu);
+void ipu_srm_dp_update(struct ipu_soc *ipu, bool sync);
int ipu_module_enable(struct ipu_soc *ipu, u32 mask);
int ipu_module_disable(struct ipu_soc *ipu, u32 mask);
int ipu_smfc_init(struct ipu_soc *ipu, struct device *dev, unsigned long base);
void ipu_smfc_exit(struct ipu_soc *ipu);
+struct ipu_pre *ipu_pre_lookup_by_phandle(struct device *dev, const char *name,
+ int index);
+int ipu_pre_get_available_count(void);
+int ipu_pre_get(struct ipu_pre *pre);
+void ipu_pre_put(struct ipu_pre *pre);
+u32 ipu_pre_get_baddr(struct ipu_pre *pre);
+void ipu_pre_configure(struct ipu_pre *pre, unsigned int width,
+ unsigned int height,
+ unsigned int stride, u32 format, unsigned int bufaddr);
+void ipu_pre_update(struct ipu_pre *pre, unsigned int bufaddr);
+
+struct ipu_prg *ipu_prg_lookup_by_phandle(struct device *dev, const char *name,
+ int ipu_id);
+
+extern struct platform_driver ipu_pre_drv;
+extern struct platform_driver ipu_prg_drv;
+
#endif /* __IPU_PRV_H__ */
* @pwr_state: current power state
* @ops: client callbacks
* @id: client identifier. Determining the id requires the handler,
- * so gpus are initially assigned VGA_SWITCHEROO_UNKNOWN_ID
- * and later given their true id in vga_switcheroo_enable()
+ * so gpus are initially assigned VGA_SWITCHEROO_UNKNOWN_ID
+ * and later given their true id in vga_switcheroo_enable()
* @active: whether the outputs are currently switched to this client
* @driver_power_control: whether power state is controlled by the driver's
- * runtime pm. If true, writing ON and OFF to the vga_switcheroo debugfs
- * interface is a no-op so as not to interfere with runtime pm
+ * runtime pm. If true, writing ON and OFF to the vga_switcheroo debugfs
+ * interface is a no-op so as not to interfere with runtime pm
* @list: client list
*
* Registered client. A client can be either a GPU or an audio device on a GPU.
/**
* struct vgasr_priv - vga_switcheroo private data
* @active: whether vga_switcheroo is enabled.
- * Prerequisite is the registration of two GPUs and a handler
+ * Prerequisite is the registration of two GPUs and a handler
* @delayed_switch_active: whether a delayed switch is pending
* @delayed_client_id: client to which a delayed switch is pending
* @debugfs_root: directory for vga_switcheroo debugfs interface
* @switch_file: file for vga_switcheroo debugfs interface
* @registered_clients: number of registered GPUs
- * (counting only vga clients, not audio clients)
+ * (counting only vga clients, not audio clients)
* @clients: list of registered clients
* @handler: registered handler
* @handler_flags: flags of registered handler
*
* Return: 0 on success, -EINVAL if a handler was already registered.
*/
-int vga_switcheroo_register_handler(const struct vga_switcheroo_handler *handler,
- enum vga_switcheroo_handler_flags_t handler_flags)
+int vga_switcheroo_register_handler(
+ const struct vga_switcheroo_handler *handler,
+ enum vga_switcheroo_handler_flags_t handler_flags)
{
mutex_lock(&vgasr_mutex);
if (vgasr_priv.handler) {
* @pdev: client pci device
* @ops: client callbacks
* @driver_power_control: whether power state is controlled by the driver's
- * runtime pm
+ * runtime pm
*
* Register vga client (GPU). Enable vga_switcheroo if another GPU and a
* handler have already registered. The power state of the client is assumed
* Return: 0 on success, -ENOMEM on memory allocation error.
*/
int vga_switcheroo_register_audio_client(struct pci_dev *pdev,
- const struct vga_switcheroo_client_ops *ops,
- enum vga_switcheroo_client_id id)
+ const struct vga_switcheroo_client_ops *ops,
+ enum vga_switcheroo_client_id id)
{
return register_client(pdev, ops, id | ID_BIT_AUDIO, false, false);
}
int ret;
/* we need to check if we have to switch back on the video
- device so the audio device can come back */
+ * device so the audio device can come back
+ */
mutex_lock(&vgasr_mutex);
list_for_each_entry(client, &vgasr_priv.clients, list) {
if (PCI_SLOT(client->pdev->devfn) == PCI_SLOT(pdev->devfn) &&
/**
* vga_switcheroo_init_domain_pm_optimus_hdmi_audio() - helper for driver
- * power control
+ * power control
* @dev: audio client device
* @domain: power domain
*
#include <linux/acpi.h>
#include <linux/i2c.h>
#include <linux/interrupt.h>
+#include <linux/pm_qos.h>
#include <asm/iosf_mbi.h>
#define SEMAPHORE_TIMEOUT 100
#define PUNIT_SEMAPHORE 0x7
+#define PUNIT_SEMAPHORE_CHT 0x10e
#define PUNIT_SEMAPHORE_BIT BIT(0)
#define PUNIT_SEMAPHORE_ACQUIRE BIT(1)
static unsigned long acquired;
-static int get_sem(struct device *dev, u32 *sem)
+static u32 get_sem_addr(struct dw_i2c_dev *dev)
{
+ if (dev->flags & MODEL_CHERRYTRAIL)
+ return PUNIT_SEMAPHORE_CHT;
+ else
+ return PUNIT_SEMAPHORE;
+}
+
+static int get_sem(struct dw_i2c_dev *dev, u32 *sem)
+{
+ u32 addr = get_sem_addr(dev);
u32 data;
int ret;
- ret = iosf_mbi_read(BT_MBI_UNIT_PMC, MBI_REG_READ, PUNIT_SEMAPHORE, &data);
+ ret = iosf_mbi_read(BT_MBI_UNIT_PMC, MBI_REG_READ, addr, &data);
if (ret) {
- dev_err(dev, "iosf failed to read punit semaphore\n");
+ dev_err(dev->dev, "iosf failed to read punit semaphore\n");
return ret;
}
return 0;
}
-static void reset_semaphore(struct device *dev)
+static void reset_semaphore(struct dw_i2c_dev *dev)
{
- u32 data;
+ if (iosf_mbi_modify(BT_MBI_UNIT_PMC, MBI_REG_READ, get_sem_addr(dev),
+ 0, PUNIT_SEMAPHORE_BIT))
+ dev_err(dev->dev, "iosf failed to reset punit semaphore during write\n");
- if (iosf_mbi_read(BT_MBI_UNIT_PMC, MBI_REG_READ, PUNIT_SEMAPHORE, &data)) {
- dev_err(dev, "iosf failed to reset punit semaphore during read\n");
- return;
- }
+ pm_qos_update_request(&dev->pm_qos, PM_QOS_DEFAULT_VALUE);
- data &= ~PUNIT_SEMAPHORE_BIT;
- if (iosf_mbi_write(BT_MBI_UNIT_PMC, MBI_REG_WRITE, PUNIT_SEMAPHORE, data))
- dev_err(dev, "iosf failed to reset punit semaphore during write\n");
+ iosf_mbi_call_pmic_bus_access_notifier_chain(MBI_PMIC_BUS_ACCESS_END,
+ NULL);
+ iosf_mbi_punit_release();
}
static int baytrail_i2c_acquire(struct dw_i2c_dev *dev)
{
+ u32 addr = get_sem_addr(dev);
u32 sem = PUNIT_SEMAPHORE_ACQUIRE;
int ret;
unsigned long start, end;
if (!dev->release_lock)
return 0;
+ iosf_mbi_punit_acquire();
+ iosf_mbi_call_pmic_bus_access_notifier_chain(MBI_PMIC_BUS_ACCESS_BEGIN,
+ NULL);
+
+ /*
+ * Disallow the CPU to enter C6 or C7 state, entering these states
+ * requires the punit to talk to the pmic and if this happens while
+ * we're holding the semaphore, the SoC hangs.
+ */
+ pm_qos_update_request(&dev->pm_qos, 0);
+
/* host driver writes to side band semaphore register */
- ret = iosf_mbi_write(BT_MBI_UNIT_PMC, MBI_REG_WRITE, PUNIT_SEMAPHORE, sem);
+ ret = iosf_mbi_write(BT_MBI_UNIT_PMC, MBI_REG_WRITE, addr, sem);
if (ret) {
dev_err(dev->dev, "iosf punit semaphore request failed\n");
- return ret;
+ goto out;
}
/* host driver waits for bit 0 to be set in semaphore register */
start = jiffies;
end = start + msecs_to_jiffies(SEMAPHORE_TIMEOUT);
do {
- ret = get_sem(dev->dev, &sem);
+ ret = get_sem(dev, &sem);
if (!ret && sem) {
acquired = jiffies;
dev_dbg(dev->dev, "punit semaphore acquired after %ums\n",
} while (time_before(jiffies, end));
dev_err(dev->dev, "punit semaphore timed out, resetting\n");
- reset_semaphore(dev->dev);
+out:
+ reset_semaphore(dev);
- ret = iosf_mbi_read(BT_MBI_UNIT_PMC, MBI_REG_READ, PUNIT_SEMAPHORE, &sem);
+ ret = iosf_mbi_read(BT_MBI_UNIT_PMC, MBI_REG_READ, addr, &sem);
if (ret)
dev_err(dev->dev, "iosf failed to read punit semaphore\n");
else
if (!dev->acquire_lock)
return;
- reset_semaphore(dev->dev);
+ reset_semaphore(dev);
dev_dbg(dev->dev, "punit semaphore held for %ums\n",
jiffies_to_msecs(jiffies - acquired));
}
-int i2c_dw_eval_lock_support(struct dw_i2c_dev *dev)
+int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
{
acpi_status status;
unsigned long long shared_host = 0;
if (ACPI_FAILURE(status))
return 0;
- if (shared_host) {
- dev_info(dev->dev, "I2C bus managed by PUNIT\n");
- dev->acquire_lock = baytrail_i2c_acquire;
- dev->release_lock = baytrail_i2c_release;
- dev->pm_runtime_disabled = true;
- }
+ if (!shared_host)
+ return 0;
if (!iosf_mbi_available())
return -EPROBE_DEFER;
+ dev_info(dev->dev, "I2C bus managed by PUNIT\n");
+ dev->acquire_lock = baytrail_i2c_acquire;
+ dev->release_lock = baytrail_i2c_release;
+ dev->pm_runtime_disabled = true;
+
+ pm_qos_add_request(&dev->pm_qos, PM_QOS_CPU_DMA_LATENCY,
+ PM_QOS_DEFAULT_VALUE);
+
return 0;
}
+
+void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev)
+{
+ if (dev->acquire_lock)
+ pm_qos_remove_request(&dev->pm_qos);
+}
{
u32 value;
- if (dev->accessor_flags & ACCESS_16BIT)
+ if (dev->flags & ACCESS_16BIT)
value = readw_relaxed(dev->base + offset) |
(readw_relaxed(dev->base + offset + 2) << 16);
else
value = readl_relaxed(dev->base + offset);
- if (dev->accessor_flags & ACCESS_SWAP)
+ if (dev->flags & ACCESS_SWAP)
return swab32(value);
else
return value;
static void dw_writel(struct dw_i2c_dev *dev, u32 b, int offset)
{
- if (dev->accessor_flags & ACCESS_SWAP)
+ if (dev->flags & ACCESS_SWAP)
b = swab32(b);
- if (dev->accessor_flags & ACCESS_16BIT) {
+ if (dev->flags & ACCESS_16BIT) {
writew_relaxed((u16)b, dev->base + offset);
writew_relaxed((u16)(b >> 16), dev->base + offset + 2);
} else {
reg = dw_readl(dev, DW_IC_COMP_TYPE);
if (reg == ___constant_swab32(DW_IC_COMP_TYPE_VALUE)) {
/* Configure register endianess access */
- dev->accessor_flags |= ACCESS_SWAP;
+ dev->flags |= ACCESS_SWAP;
} else if (reg == (DW_IC_COMP_TYPE_VALUE & 0x0000ffff)) {
/* Configure register access mode 16bit */
- dev->accessor_flags |= ACCESS_16BIT;
+ dev->flags |= ACCESS_16BIT;
} else if (reg != DW_IC_COMP_TYPE_VALUE) {
dev_err(dev->dev, "Unknown Synopsys component type: "
"0x%08x\n", reg);
tx_aborted:
if ((stat & (DW_IC_INTR_TX_ABRT | DW_IC_INTR_STOP_DET)) || dev->msg_err)
complete(&dev->cmd_complete);
- else if (unlikely(dev->accessor_flags & ACCESS_INTR_MASK)) {
+ else if (unlikely(dev->flags & ACCESS_INTR_MASK)) {
/* workaround to trigger pending interrupt */
stat = dw_readl(dev, DW_IC_INTR_MASK);
i2c_dw_disable_int(dev);
*/
#include <linux/i2c.h>
+#include <linux/pm_qos.h>
#define DW_IC_DEFAULT_FUNCTIONALITY (I2C_FUNC_I2C | \
I2C_FUNC_SMBUS_BYTE | \
* @fp_lcnt: fast plus LCNT value
* @hs_hcnt: high speed HCNT value
* @hs_lcnt: high speed LCNT value
+ * @pm_qos: pm_qos_request used while holding a hardware lock on the bus
* @acquire_lock: function to acquire a hardware lock on the bus
* @release_lock: function to release a hardware lock on the bus
* @pm_runtime_disabled: true if pm runtime is disabled
unsigned int status;
u32 abort_source;
int irq;
- u32 accessor_flags;
+ u32 flags;
struct i2c_adapter adapter;
u32 functionality;
u32 master_cfg;
u16 fp_lcnt;
u16 hs_hcnt;
u16 hs_lcnt;
+ struct pm_qos_request pm_qos;
int (*acquire_lock)(struct dw_i2c_dev *dev);
void (*release_lock)(struct dw_i2c_dev *dev);
bool pm_runtime_disabled;
#define ACCESS_16BIT 0x00000002
#define ACCESS_INTR_MASK 0x00000004
+#define MODEL_CHERRYTRAIL 0x00000100
+
extern int i2c_dw_init(struct dw_i2c_dev *dev);
extern void i2c_dw_disable(struct dw_i2c_dev *dev);
extern void i2c_dw_disable_int(struct dw_i2c_dev *dev);
extern int i2c_dw_probe(struct dw_i2c_dev *dev);
#if IS_ENABLED(CONFIG_I2C_DESIGNWARE_BAYTRAIL)
-extern int i2c_dw_eval_lock_support(struct dw_i2c_dev *dev);
+extern int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev);
+extern void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev);
#else
-static inline int i2c_dw_eval_lock_support(struct dw_i2c_dev *dev) { return 0; }
+static inline int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) { return 0; }
+static inline void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev) {}
#endif
medfield,
merrifield,
baytrail,
+ cherrytrail,
haswell,
};
u32 rx_fifo_depth;
u32 clk_khz;
u32 functionality;
+ u32 flags;
struct dw_scl_sda_cfg *scl_sda_cfg;
int (*setup)(struct pci_dev *pdev, struct dw_pci_controller *c);
};
.functionality = I2C_FUNC_10BIT_ADDR,
.scl_sda_cfg = &hsw_config,
},
+ [cherrytrail] = {
+ .bus_num = -1,
+ .bus_cfg = INTEL_MID_STD_CFG | DW_IC_CON_SPEED_FAST,
+ .tx_fifo_depth = 32,
+ .rx_fifo_depth = 32,
+ .functionality = I2C_FUNC_10BIT_ADDR,
+ .flags = MODEL_CHERRYTRAIL,
+ .scl_sda_cfg = &byt_config,
+ },
};
#ifdef CONFIG_PM
dev->base = pcim_iomap_table(pdev)[0];
dev->dev = &pdev->dev;
dev->irq = pdev->irq;
+ dev->flags |= controller->flags;
if (controller->setup) {
r = controller->setup(pdev, controller);
{ PCI_VDEVICE(INTEL, 0x9c61), haswell },
{ PCI_VDEVICE(INTEL, 0x9c62), haswell },
/* Braswell / Cherrytrail */
- { PCI_VDEVICE(INTEL, 0x22C1), baytrail },
- { PCI_VDEVICE(INTEL, 0x22C2), baytrail },
- { PCI_VDEVICE(INTEL, 0x22C3), baytrail },
- { PCI_VDEVICE(INTEL, 0x22C4), baytrail },
- { PCI_VDEVICE(INTEL, 0x22C5), baytrail },
- { PCI_VDEVICE(INTEL, 0x22C6), baytrail },
- { PCI_VDEVICE(INTEL, 0x22C7), baytrail },
+ { PCI_VDEVICE(INTEL, 0x22C1), cherrytrail },
+ { PCI_VDEVICE(INTEL, 0x22C2), cherrytrail },
+ { PCI_VDEVICE(INTEL, 0x22C3), cherrytrail },
+ { PCI_VDEVICE(INTEL, 0x22C4), cherrytrail },
+ { PCI_VDEVICE(INTEL, 0x22C5), cherrytrail },
+ { PCI_VDEVICE(INTEL, 0x22C6), cherrytrail },
+ { PCI_VDEVICE(INTEL, 0x22C7), cherrytrail },
{ 0,}
};
MODULE_DEVICE_TABLE(pci, i2_designware_pci_ids);
id = acpi_match_device(pdev->dev.driver->acpi_match_table, &pdev->dev);
if (id && id->driver_data)
- dev->accessor_flags |= (u32)id->driver_data;
+ dev->flags |= (u32)id->driver_data;
return 0;
}
{ "INT3432", 0 },
{ "INT3433", 0 },
{ "80860F41", 0 },
- { "808622C1", 0 },
+ { "808622C1", MODEL_CHERRYTRAIL },
{ "AMD0010", ACCESS_INTR_MASK },
{ "AMDI0010", ACCESS_INTR_MASK },
{ "AMDI0510", 0 },
goto exit_reset;
}
- r = i2c_dw_eval_lock_support(dev);
+ r = i2c_dw_probe_lock_support(dev);
if (r)
goto exit_reset;
if (!IS_ERR_OR_NULL(dev->rst))
reset_control_assert(dev->rst);
+ i2c_dw_remove_lock_support(dev);
+
return 0;
}
}
EXPORT_SYMBOL_GPL(of_platform_depopulate);
+static void devm_of_platform_populate_release(struct device *dev, void *res)
+{
+ of_platform_depopulate(*(struct device **)res);
+}
+
+/**
+ * devm_of_platform_populate() - Populate platform_devices from device tree data
+ * @dev: device that requested to populate from device tree data
+ *
+ * Similar to of_platform_populate(), but will automatically call
+ * of_platform_depopulate() when the device is unbound from the bus.
+ *
+ * Returns 0 on success, < 0 on failure.
+ */
+int devm_of_platform_populate(struct device *dev)
+{
+ struct device **ptr;
+ int ret;
+
+ if (!dev)
+ return -EINVAL;
+
+ ptr = devres_alloc(devm_of_platform_populate_release,
+ sizeof(*ptr), GFP_KERNEL);
+ if (!ptr)
+ return -ENOMEM;
+
+ ret = of_platform_populate(dev->of_node, NULL, NULL, dev);
+ if (ret) {
+ devres_free(ptr);
+ } else {
+ *ptr = dev;
+ devres_add(dev, ptr);
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(devm_of_platform_populate);
+
+static int devm_of_platform_match(struct device *dev, void *res, void *data)
+{
+ struct device **ptr = res;
+
+ if (!ptr) {
+ WARN_ON(!ptr);
+ return 0;
+ }
+
+ return *ptr == data;
+}
+
+/**
+ * devm_of_platform_depopulate() - Remove devices populated from device tree
+ * @dev: device that requested to depopulate from device tree data
+ *
+ * Complementary to devm_of_platform_populate(), this function removes children
+ * of the given device (and, recurrently, their children) that have been
+ * created from their respective device tree nodes (and only those,
+ * leaving others - eg. manually created - unharmed).
+ */
+void devm_of_platform_depopulate(struct device *dev)
+{
+ int ret;
+
+ ret = devres_release(dev, devm_of_platform_populate_release,
+ devm_of_platform_match, dev);
+
+ WARN_ON(ret);
+}
+EXPORT_SYMBOL_GPL(devm_of_platform_depopulate);
+
#ifdef CONFIG_OF_DYNAMIC
static int of_platform_notify(struct notifier_block *nb,
unsigned long action, void *arg)
struct analogix_dp_plat_data *plat_data);
void analogix_dp_unbind(struct device *dev, struct device *master, void *data);
+int analogix_dp_start_crc(struct drm_connector *connector);
+int analogix_dp_stop_crc(struct drm_connector *connector);
+
#endif /* _ANALOGIX_DP_H_ */
DW_HDMI_RES_MAX,
};
-enum dw_hdmi_devtype {
- IMX6Q_HDMI,
- IMX6DL_HDMI,
- RK3288_HDMI,
-};
-
enum dw_hdmi_phy_type {
DW_HDMI_PHY_DWC_HDMI_TX_PHY = 0x00,
DW_HDMI_PHY_DWC_MHL_PHY_HEAC = 0xb2,
u16 vlev_ctr; /* voltage level control */
};
+struct dw_hdmi_phy_ops {
+ int (*init)(struct dw_hdmi *hdmi, void *data,
+ struct drm_display_mode *mode);
+ void (*disable)(struct dw_hdmi *hdmi, void *data);
+ enum drm_connector_status (*read_hpd)(struct dw_hdmi *hdmi, void *data);
+};
+
struct dw_hdmi_plat_data {
- enum dw_hdmi_devtype dev_type;
+ struct regmap *regm;
+ enum drm_mode_status (*mode_valid)(struct drm_connector *connector,
+ struct drm_display_mode *mode);
+
+ /* Vendor PHY support */
+ const struct dw_hdmi_phy_ops *phy_ops;
+ const char *phy_name;
+ void *phy_data;
+
+ /* Synopsys PHY support */
const struct dw_hdmi_mpll_config *mpll_cfg;
const struct dw_hdmi_curr_ctrl *cur_ctr;
const struct dw_hdmi_phy_config *phy_config;
- enum drm_mode_status (*mode_valid)(struct drm_connector *connector,
- struct drm_display_mode *mode);
+ int (*configure_phy)(struct dw_hdmi *hdmi,
+ const struct dw_hdmi_plat_data *pdata,
+ unsigned long mpixelclock);
};
int dw_hdmi_probe(struct platform_device *pdev,
void dw_hdmi_audio_enable(struct dw_hdmi *hdmi);
void dw_hdmi_audio_disable(struct dw_hdmi *hdmi);
+/* PHY configuration */
+void dw_hdmi_phy_i2c_write(struct dw_hdmi *hdmi, unsigned short data,
+ unsigned char addr);
+
#endif /* __IMX_HDMI_H__ */
#include <linux/miscdevice.h>
#include <linux/mm.h>
#include <linux/mutex.h>
-#include <linux/pci.h>
#include <linux/platform_device.h>
#include <linux/poll.h>
#include <linux/ratelimit.h>
-#include <linux/rbtree.h>
#include <linux/sched.h>
#include <linux/slab.h>
#include <linux/types.h>
#include <linux/vmalloc.h>
#include <linux/workqueue.h>
#include <linux/dma-fence.h>
+#include <linux/module.h>
#include <asm/mman.h>
#include <asm/pgalloc.h>
#include <drm/drm_mm.h>
#include <drm/drm_os_linux.h>
#include <drm/drm_sarea.h>
-#include <drm/drm_vma_manager.h>
#include <drm/drm_drv.h>
+#include <drm/drm_prime.h>
+#include <drm/drm_pci.h>
+#include <drm/drm_file.h>
struct module;
-struct drm_file;
struct drm_device;
struct drm_agp_head;
struct drm_local_map;
struct drm_device_dma;
-struct drm_dma_handle;
struct drm_gem_object;
struct drm_master;
struct drm_vblank_crtc;
+struct drm_vma_offset_manager;
struct device_node;
struct videomode;
struct reservation_object;
struct dma_buf_attachment;
+struct pci_dev;
+struct pci_controller;
+
/*
* The following categories are defined:
*
.name = #ioctl \
}
-/* Event queued up for userspace to read */
-struct drm_pending_event {
- struct completion *completion;
- void (*completion_release)(struct completion *completion);
- struct drm_event *event;
- struct dma_fence *fence;
- struct list_head link;
- struct list_head pending_link;
- struct drm_file *file_priv;
- pid_t pid; /* pid of requester, no guarantee it's valid by the time
- we deliver the event, for tracing only */
-};
-
-struct drm_prime_file_private {
- struct mutex lock;
- struct rb_root dmabufs;
- struct rb_root handles;
-};
-
-/** File private data */
-struct drm_file {
- unsigned authenticated :1;
- /* true when the client has asked us to expose stereo 3D mode flags */
- unsigned stereo_allowed :1;
- /*
- * true if client understands CRTC primary planes and cursor planes
- * in the plane list
- */
- unsigned universal_planes:1;
- /* true if client understands atomic properties */
- unsigned atomic:1;
- /*
- * This client is the creator of @master.
- * Protected by struct drm_device::master_mutex.
- */
- unsigned is_master:1;
-
- struct pid *pid;
- drm_magic_t magic;
- struct list_head lhead;
- struct drm_minor *minor;
- unsigned long lock_count;
-
- /** Mapping of mm object handles to object pointers. */
- struct idr object_idr;
- /** Lock for synchronization of access to object_idr. */
- spinlock_t table_lock;
-
- struct file *filp;
- void *driver_priv;
-
- struct drm_master *master; /* master this node is currently associated with
- N.B. not always dev->master */
- /**
- * fbs - List of framebuffers associated with this file.
- *
- * Protected by fbs_lock. Note that the fbs list holds a reference on
- * the fb object to prevent it from untimely disappearing.
- */
- struct list_head fbs;
- struct mutex fbs_lock;
-
- /** User-created blob properties; this retains a reference on the
- * property. */
- struct list_head blobs;
-
- wait_queue_head_t event_wait;
- struct list_head pending_event_list;
- struct list_head event_list;
- int event_space;
-
- struct mutex event_read_lock;
-
- struct drm_prime_file_private prime;
-};
-
-/**
- * Lock data.
- */
-struct drm_lock_data {
- struct drm_hw_lock *hw_lock; /**< Hardware lock */
- /** Private of lock holder's file (NULL=kernel) */
- struct drm_file *file_priv;
- wait_queue_head_t lock_queue; /**< Queue of blocked processes */
- unsigned long lock_time; /**< Time of last lock in jiffies */
- spinlock_t spinlock;
- uint32_t kernel_waiters;
- uint32_t user_waiters;
- int idle_has_lock;
-};
-
/* Flags and return codes for get_vblank_timestamp() driver function. */
#define DRM_CALLED_FROM_VBLIRQ 1
#define DRM_VBLANKTIME_SCANOUTPOS_METHOD (1 << 0)
#define DRM_SCANOUTPOS_IN_VBLANK (1 << 1)
#define DRM_SCANOUTPOS_ACCURATE (1 << 2)
-enum drm_minor_type {
- DRM_MINOR_PRIMARY,
- DRM_MINOR_CONTROL,
- DRM_MINOR_RENDER,
- DRM_MINOR_CNT,
-};
-
/**
* Info file list entry. This structure represents a debugfs or proc file to
* be created by the drm core
struct dentry *dent;
};
-/**
- * DRM minor structure. This structure represents a drm minor number.
- */
-struct drm_minor {
- int index; /**< Minor device number */
- int type; /**< Control or render */
- struct device *kdev; /**< Linux device */
- struct drm_device *dev;
-
- struct dentry *debugfs_root;
-
- struct list_head debugfs_list;
- struct mutex debugfs_lock; /* Protects debugfs_list. */
-};
-
/**
* DRM device structure. This structure represent a complete card that
* may contain multiple heads.
struct pci_controller *hose;
#endif
- struct platform_device *platformdev; /**< Platform device struture */
struct virtio_device *virtdev;
struct drm_sg_mem *sg; /**< Scatter gather memory */
return ret;
}
-static inline bool drm_is_render_client(const struct drm_file *file_priv)
-{
- return file_priv->minor->type == DRM_MINOR_RENDER;
-}
-
-static inline bool drm_is_control_client(const struct drm_file *file_priv)
-{
- return file_priv->minor->type == DRM_MINOR_CONTROL;
-}
-
-static inline bool drm_is_primary_client(const struct drm_file *file_priv)
-{
- return file_priv->minor->type == DRM_MINOR_PRIMARY;
-}
-
/******************************************************************/
/** \name Internal function definitions */
/*@{*/
#endif
extern bool drm_ioctl_flags(unsigned int nr, unsigned int *flags);
-/* File Operations (drm_fops.c) */
-int drm_open(struct inode *inode, struct file *filp);
-ssize_t drm_read(struct file *filp, char __user *buffer,
- size_t count, loff_t *offset);
-int drm_release(struct inode *inode, struct file *filp);
-unsigned int drm_poll(struct file *filp, struct poll_table_struct *wait);
-int drm_event_reserve_init_locked(struct drm_device *dev,
- struct drm_file *file_priv,
- struct drm_pending_event *p,
- struct drm_event *e);
-int drm_event_reserve_init(struct drm_device *dev,
- struct drm_file *file_priv,
- struct drm_pending_event *p,
- struct drm_event *e);
-void drm_event_cancel_free(struct drm_device *dev,
- struct drm_pending_event *p);
-void drm_send_event_locked(struct drm_device *dev, struct drm_pending_event *e);
-void drm_send_event(struct drm_device *dev, struct drm_pending_event *e);
-
/* Misc. IOCTL support (drm_ioctl.c) */
int drm_noop(struct drm_device *dev, void *data,
struct drm_file *file_priv);
}
#endif
-struct dma_buf_export_info;
-
-extern struct dma_buf *drm_gem_prime_export(struct drm_device *dev,
- struct drm_gem_object *obj,
- int flags);
-extern int drm_gem_prime_handle_to_fd(struct drm_device *dev,
- struct drm_file *file_priv, uint32_t handle, uint32_t flags,
- int *prime_fd);
-extern struct drm_gem_object *drm_gem_prime_import(struct drm_device *dev,
- struct dma_buf *dma_buf);
-extern int drm_gem_prime_fd_to_handle(struct drm_device *dev,
- struct drm_file *file_priv, int prime_fd, uint32_t *handle);
-struct dma_buf *drm_gem_dmabuf_export(struct drm_device *dev,
- struct dma_buf_export_info *exp_info);
-extern void drm_gem_dmabuf_release(struct dma_buf *dma_buf);
-
-extern int drm_prime_sg_to_page_addr_arrays(struct sg_table *sgt, struct page **pages,
- dma_addr_t *addrs, int max_pages);
-extern struct sg_table *drm_prime_pages_to_sg(struct page **pages, unsigned int nr_pages);
-extern void drm_prime_gem_destroy(struct drm_gem_object *obj, struct sg_table *sg);
-
-
-extern struct drm_dma_handle *drm_pci_alloc(struct drm_device *dev, size_t size,
- size_t align);
-extern void drm_pci_free(struct drm_device *dev, struct drm_dma_handle * dmah);
-
/* sysfs support (drm_sysfs.c) */
extern void drm_sysfs_hotplug_event(struct drm_device *dev);
/*@}*/
-extern int drm_pci_init(struct drm_driver *driver, struct pci_driver *pdriver);
-extern void drm_pci_exit(struct drm_driver *driver, struct pci_driver *pdriver);
-#ifdef CONFIG_PCI
-extern int drm_get_pci_dev(struct pci_dev *pdev,
- const struct pci_device_id *ent,
- struct drm_driver *driver);
-extern int drm_pci_set_busid(struct drm_device *dev, struct drm_master *master);
-#else
-static inline int drm_get_pci_dev(struct pci_dev *pdev,
- const struct pci_device_id *ent,
- struct drm_driver *driver)
-{
- return -ENOSYS;
-}
-
-static inline int drm_pci_set_busid(struct drm_device *dev,
- struct drm_master *master)
-{
- return -ENOSYS;
-}
-#endif
-
-#define DRM_PCIE_SPEED_25 1
-#define DRM_PCIE_SPEED_50 2
-#define DRM_PCIE_SPEED_80 4
-
-extern int drm_pcie_get_speed_cap_mask(struct drm_device *dev, u32 *speed_mask);
-extern int drm_pcie_get_max_link_width(struct drm_device *dev, u32 *mlw);
-
-/* platform section */
-extern int drm_platform_init(struct drm_driver *driver, struct platform_device *platform_device);
-
/* returns true if currently okay to sleep */
static __inline__ bool drm_can_sleep(void)
{
struct __drm_planes_state {
struct drm_plane *ptr;
- struct drm_plane_state *state;
+ struct drm_plane_state *state, *old_state, *new_state;
};
struct __drm_crtcs_state {
struct drm_crtc *ptr;
- struct drm_crtc_state *state;
+ struct drm_crtc_state *state, *old_state, *new_state;
struct drm_crtc_commit *commit;
s32 __user *out_fence_ptr;
unsigned last_vblank_count;
struct __drm_connnectors_state {
struct drm_connector *ptr;
- struct drm_connector_state *state;
+ struct drm_connector_state *state, *old_state, *new_state;
};
/**
*
* This function returns the crtc state for the given crtc, or NULL
* if the crtc is not part of the global atomic state.
+ *
+ * This function is deprecated, @drm_atomic_get_old_crtc_state or
+ * @drm_atomic_get_new_crtc_state should be used instead.
*/
static inline struct drm_crtc_state *
drm_atomic_get_existing_crtc_state(struct drm_atomic_state *state,
return state->crtcs[drm_crtc_index(crtc)].state;
}
+/**
+ * drm_atomic_get_old_crtc_state - get old crtc state, if it exists
+ * @state: global atomic state object
+ * @crtc: crtc to grab
+ *
+ * This function returns the old crtc state for the given crtc, or
+ * NULL if the crtc is not part of the global atomic state.
+ */
+static inline struct drm_crtc_state *
+drm_atomic_get_old_crtc_state(struct drm_atomic_state *state,
+ struct drm_crtc *crtc)
+{
+ return state->crtcs[drm_crtc_index(crtc)].old_state;
+}
+/**
+ * drm_atomic_get_new_crtc_state - get new crtc state, if it exists
+ * @state: global atomic state object
+ * @crtc: crtc to grab
+ *
+ * This function returns the new crtc state for the given crtc, or
+ * NULL if the crtc is not part of the global atomic state.
+ */
+static inline struct drm_crtc_state *
+drm_atomic_get_new_crtc_state(struct drm_atomic_state *state,
+ struct drm_crtc *crtc)
+{
+ return state->crtcs[drm_crtc_index(crtc)].new_state;
+}
+
/**
* drm_atomic_get_existing_plane_state - get plane state, if it exists
* @state: global atomic state object
*
* This function returns the plane state for the given plane, or NULL
* if the plane is not part of the global atomic state.
+ *
+ * This function is deprecated, @drm_atomic_get_old_plane_state or
+ * @drm_atomic_get_new_plane_state should be used instead.
*/
static inline struct drm_plane_state *
drm_atomic_get_existing_plane_state(struct drm_atomic_state *state,
return state->planes[drm_plane_index(plane)].state;
}
+/**
+ * drm_atomic_get_old_plane_state - get plane state, if it exists
+ * @state: global atomic state object
+ * @plane: plane to grab
+ *
+ * This function returns the old plane state for the given plane, or
+ * NULL if the plane is not part of the global atomic state.
+ */
+static inline struct drm_plane_state *
+drm_atomic_get_old_plane_state(struct drm_atomic_state *state,
+ struct drm_plane *plane)
+{
+ return state->planes[drm_plane_index(plane)].old_state;
+}
+
+/**
+ * drm_atomic_get_new_plane_state - get plane state, if it exists
+ * @state: global atomic state object
+ * @plane: plane to grab
+ *
+ * This function returns the new plane state for the given plane, or
+ * NULL if the plane is not part of the global atomic state.
+ */
+static inline struct drm_plane_state *
+drm_atomic_get_new_plane_state(struct drm_atomic_state *state,
+ struct drm_plane *plane)
+{
+ return state->planes[drm_plane_index(plane)].new_state;
+}
+
/**
* drm_atomic_get_existing_connector_state - get connector state, if it exists
* @state: global atomic state object
*
* This function returns the connector state for the given connector,
* or NULL if the connector is not part of the global atomic state.
+ *
+ * This function is deprecated, @drm_atomic_get_old_connector_state or
+ * @drm_atomic_get_new_connector_state should be used instead.
*/
static inline struct drm_connector_state *
drm_atomic_get_existing_connector_state(struct drm_atomic_state *state,
return state->connectors[index].state;
}
+/**
+ * drm_atomic_get_old_connector_state - get connector state, if it exists
+ * @state: global atomic state object
+ * @connector: connector to grab
+ *
+ * This function returns the old connector state for the given connector,
+ * or NULL if the connector is not part of the global atomic state.
+ */
+static inline struct drm_connector_state *
+drm_atomic_get_old_connector_state(struct drm_atomic_state *state,
+ struct drm_connector *connector)
+{
+ int index = drm_connector_index(connector);
+
+ if (index >= state->num_connector)
+ return NULL;
+
+ return state->connectors[index].old_state;
+}
+
+/**
+ * drm_atomic_get_new_connector_state - get connector state, if it exists
+ * @state: global atomic state object
+ * @connector: connector to grab
+ *
+ * This function returns the new connector state for the given connector,
+ * or NULL if the connector is not part of the global atomic state.
+ */
+static inline struct drm_connector_state *
+drm_atomic_get_new_connector_state(struct drm_atomic_state *state,
+ struct drm_connector *connector)
+{
+ int index = drm_connector_index(connector);
+
+ if (index >= state->num_connector)
+ return NULL;
+
+ return state->connectors[index].new_state;
+}
+
/**
* __drm_atomic_get_current_plane_state - get current plane state
* @state: global atomic state object
(__i)++) \
for_each_if (connector)
+#define for_each_oldnew_connector_in_state(__state, connector, old_connector_state, new_connector_state, __i) \
+ for ((__i) = 0; \
+ (__i) < (__state)->num_connector && \
+ ((connector) = (__state)->connectors[__i].ptr, \
+ (old_connector_state) = (__state)->connectors[__i].old_state, \
+ (new_connector_state) = (__state)->connectors[__i].new_state, 1); \
+ (__i)++) \
+ for_each_if (connector)
+
+#define for_each_old_connector_in_state(__state, connector, old_connector_state, __i) \
+ for ((__i) = 0; \
+ (__i) < (__state)->num_connector && \
+ ((connector) = (__state)->connectors[__i].ptr, \
+ (old_connector_state) = (__state)->connectors[__i].old_state, 1); \
+ (__i)++) \
+ for_each_if (connector)
+
+#define for_each_new_connector_in_state(__state, connector, new_connector_state, __i) \
+ for ((__i) = 0; \
+ (__i) < (__state)->num_connector && \
+ ((connector) = (__state)->connectors[__i].ptr, \
+ (new_connector_state) = (__state)->connectors[__i].new_state, 1); \
+ (__i)++) \
+ for_each_if (connector)
+
#define for_each_crtc_in_state(__state, crtc, crtc_state, __i) \
for ((__i) = 0; \
(__i) < (__state)->dev->mode_config.num_crtc && \
(__i)++) \
for_each_if (crtc_state)
+#define for_each_oldnew_crtc_in_state(__state, crtc, old_crtc_state, new_crtc_state, __i) \
+ for ((__i) = 0; \
+ (__i) < (__state)->dev->mode_config.num_crtc && \
+ ((crtc) = (__state)->crtcs[__i].ptr, \
+ (old_crtc_state) = (__state)->crtcs[__i].old_state, \
+ (new_crtc_state) = (__state)->crtcs[__i].new_state, 1); \
+ (__i)++) \
+ for_each_if (crtc)
+
+#define for_each_old_crtc_in_state(__state, crtc, old_crtc_state, __i) \
+ for ((__i) = 0; \
+ (__i) < (__state)->dev->mode_config.num_crtc && \
+ ((crtc) = (__state)->crtcs[__i].ptr, \
+ (old_crtc_state) = (__state)->crtcs[__i].old_state, 1); \
+ (__i)++) \
+ for_each_if (crtc)
+
+#define for_each_new_crtc_in_state(__state, crtc, new_crtc_state, __i) \
+ for ((__i) = 0; \
+ (__i) < (__state)->dev->mode_config.num_crtc && \
+ ((crtc) = (__state)->crtcs[__i].ptr, \
+ (new_crtc_state) = (__state)->crtcs[__i].new_state, 1); \
+ (__i)++) \
+ for_each_if (crtc)
+
#define for_each_plane_in_state(__state, plane, plane_state, __i) \
for ((__i) = 0; \
(__i) < (__state)->dev->mode_config.num_total_plane && \
(__i)++) \
for_each_if (plane_state)
+#define for_each_oldnew_plane_in_state(__state, plane, old_plane_state, new_plane_state, __i) \
+ for ((__i) = 0; \
+ (__i) < (__state)->dev->mode_config.num_total_plane && \
+ ((plane) = (__state)->planes[__i].ptr, \
+ (old_plane_state) = (__state)->planes[__i].old_state, \
+ (new_plane_state) = (__state)->planes[__i].new_state, 1); \
+ (__i)++) \
+ for_each_if (plane)
+
+#define for_each_old_plane_in_state(__state, plane, old_plane_state, __i) \
+ for ((__i) = 0; \
+ (__i) < (__state)->dev->mode_config.num_total_plane && \
+ ((plane) = (__state)->planes[__i].ptr, \
+ (old_plane_state) = (__state)->planes[__i].old_state, 1); \
+ (__i)++) \
+ for_each_if (plane)
+
+#define for_each_new_plane_in_state(__state, plane, new_plane_state, __i) \
+ for ((__i) = 0; \
+ (__i) < (__state)->dev->mode_config.num_total_plane && \
+ ((plane) = (__state)->planes[__i].ptr, \
+ (new_plane_state) = (__state)->planes[__i].new_state, 1); \
+ (__i)++) \
+ for_each_if (plane)
+
/**
* drm_atomic_crtc_needs_modeset - compute combined modeset need
* @state: &drm_crtc_state for the CRTC
int drm_atomic_helper_disable_all(struct drm_device *dev,
struct drm_modeset_acquire_ctx *ctx);
struct drm_atomic_state *drm_atomic_helper_suspend(struct drm_device *dev);
+int drm_atomic_helper_commit_duplicated_state(struct drm_atomic_state *state,
+ struct drm_modeset_acquire_ctx *ctx);
int drm_atomic_helper_resume(struct drm_device *dev,
struct drm_atomic_state *state);
__drm_atomic_get_current_plane_state((crtc_state)->state, \
plane)))
-/*
+/**
* drm_atomic_plane_disabling - check whether a plane is being disabled
- * @plane: plane object
- * @old_state: previous atomic state
+ * @old_plane_state: old atomic plane state
+ * @new_plane_state: new atomic plane state
*
* Checks the atomic state of a plane to determine whether it's being disabled
* or not. This also WARNs if it detects an invalid state (both CRTC and FB
* True if the plane is being disabled, false otherwise.
*/
static inline bool
-drm_atomic_plane_disabling(struct drm_plane *plane,
- struct drm_plane_state *old_state)
+drm_atomic_plane_disabling(struct drm_plane_state *old_plane_state,
+ struct drm_plane_state *new_plane_state)
{
/*
* When disabling a plane, CRTC and FB should always be NULL together.
* Anything else should be considered a bug in the atomic core, so we
* gently warn about it.
*/
- WARN_ON((plane->state->crtc == NULL && plane->state->fb != NULL) ||
- (plane->state->crtc != NULL && plane->state->fb == NULL));
+ WARN_ON((new_plane_state->crtc == NULL && new_plane_state->fb != NULL) ||
+ (new_plane_state->crtc != NULL && new_plane_state->fb == NULL));
- /*
- * When using the transitional helpers, old_state may be NULL. If so,
- * we know nothing about the current state and have to assume that it
- * might be enabled.
- *
- * When using the atomic helpers, old_state won't be NULL. Therefore
- * this check assumes that either the driver will have reconstructed
- * the correct state in ->reset() or that the driver will have taken
- * appropriate measures to disable all planes.
- */
- return (!old_state || old_state->crtc) && !plane->state->crtc;
+ return old_plane_state->crtc && !new_plane_state->crtc;
}
#endif /* DRM_ATOMIC_HELPER_H_ */
#ifndef _DRM_AUTH_H_
#define _DRM_AUTH_H_
+/*
+ * Legacy DRI1 locking data structure. Only here instead of in drm_legacy.h for
+ * include ordering reasons.
+ *
+ * DO NOT USE.
+ */
+struct drm_lock_data {
+ struct drm_hw_lock *hw_lock;
+ struct drm_file *file_priv;
+ wait_queue_head_t lock_queue;
+ unsigned long lock_time;
+ spinlock_t spinlock;
+ uint32_t kernel_waiters;
+ uint32_t user_waiters;
+ int idle_has_lock;
+};
+
/**
* struct drm_master - drm master structure
*
SubPixelVerticalRGB,
SubPixelVerticalBGR,
SubPixelNone,
+
+};
+
+/**
+ * struct drm_scrambling: sink's scrambling support.
+ */
+struct drm_scrambling {
+ /**
+ * @supported: scrambling supported for rates > 340 Mhz.
+ */
+ bool supported;
+ /**
+ * @low_rates: scrambling supported for rates <= 340 Mhz.
+ */
+ bool low_rates;
+};
+
+/*
+ * struct drm_scdc - Information about scdc capabilities of a HDMI 2.0 sink
+ *
+ * Provides SCDC register support and capabilities related information on a
+ * HDMI 2.0 sink. In case of a HDMI 1.4 sink, all parameter must be 0.
+ */
+struct drm_scdc {
+ /**
+ * @supported: status control & data channel present.
+ */
+ bool supported;
+ /**
+ * @read_request: sink is capable of generating scdc read request.
+ */
+ bool read_request;
+ /**
+ * @scrambling: sink's scrambling capabilities
+ */
+ struct drm_scrambling scrambling;
+};
+
+
+/**
+ * struct drm_hdmi_info - runtime information about the connected HDMI sink
+ *
+ * Describes if a given display supports advanced HDMI 2.0 features.
+ * This information is available in CEA-861-F extension blocks (like HF-VSDB).
+ */
+struct drm_hdmi_info {
+ struct drm_scdc scdc;
+};
+
+/**
+ * enum drm_link_status - connector's link_status property value
+ *
+ * This enum is used as the connector's link status property value.
+ * It is set to the values defined in uapi.
+ *
+ * @DRM_LINK_STATUS_GOOD: DP Link is Good as a result of successful
+ * link training
+ * @DRM_LINK_STATUS_BAD: DP Link is BAD as a result of link training
+ * failure
+ */
+enum drm_link_status {
+ DRM_LINK_STATUS_GOOD = DRM_MODE_LINK_STATUS_GOOD,
+ DRM_LINK_STATUS_BAD = DRM_MODE_LINK_STATUS_BAD,
};
/**
* @cea_rev: CEA revision of the HDMI sink.
*/
u8 cea_rev;
+
+ /**
+ * @hdmi: advance features of a HDMI sink.
+ */
+ struct drm_hdmi_info hdmi;
};
int drm_display_info_set_bus_formats(struct drm_display_info *info,
struct drm_encoder *best_encoder;
+ /**
+ * @link_status: Connector link_status to keep track of whether link is
+ * GOOD or BAD to notify userspace if retraining is necessary.
+ */
+ enum drm_link_status link_status;
+
struct drm_atomic_state *state;
struct drm_tv_connector_state tv;
}
/**
- * drm_connector_reference - incr the connector refcnt
- * @connector: connector
+ * drm_connector_get - acquire a connector reference
+ * @connector: DRM connector
*
* This function increments the connector's refcount.
*/
+static inline void drm_connector_get(struct drm_connector *connector)
+{
+ drm_mode_object_get(&connector->base);
+}
+
+/**
+ * drm_connector_put - release a connector reference
+ * @connector: DRM connector
+ *
+ * This function decrements the connector's reference count and frees the
+ * object if the reference count drops to zero.
+ */
+static inline void drm_connector_put(struct drm_connector *connector)
+{
+ drm_mode_object_put(&connector->base);
+}
+
+/**
+ * drm_connector_reference - acquire a connector reference
+ * @connector: DRM connector
+ *
+ * This is a compatibility alias for drm_connector_get() and should not be
+ * used by new code.
+ */
static inline void drm_connector_reference(struct drm_connector *connector)
{
- drm_mode_object_reference(&connector->base);
+ drm_connector_get(connector);
}
/**
- * drm_connector_unreference - unref a connector
- * @connector: connector to unref
+ * drm_connector_unreference - release a connector reference
+ * @connector: DRM connector
*
- * This function decrements the connector's refcount and frees it if it drops to zero.
+ * This is a compatibility alias for drm_connector_put() and should not be
+ * used by new code.
*/
static inline void drm_connector_unreference(struct drm_connector *connector)
{
- drm_mode_object_unreference(&connector->base);
+ drm_connector_put(connector);
}
const char *drm_get_connector_status_name(enum drm_connector_status status);
int drm_mode_connector_set_tile_property(struct drm_connector *connector);
int drm_mode_connector_update_edid_property(struct drm_connector *connector,
const struct edid *edid);
+void drm_mode_connector_set_link_status_property(struct drm_connector *connector,
+ uint64_t link_status);
/**
* struct drm_tile_group - Tile group metadata
*
* This iterator tracks state needed to be able to walk the connector_list
* within struct drm_mode_config. Only use together with
- * drm_connector_list_iter_get(), drm_connector_list_iter_put() and
+ * drm_connector_list_iter_begin(), drm_connector_list_iter_end() and
* drm_connector_list_iter_next() respectively the convenience macro
* drm_for_each_connector_iter().
*/
struct drm_connector *conn;
};
-void drm_connector_list_iter_get(struct drm_device *dev,
- struct drm_connector_list_iter *iter);
+void drm_connector_list_iter_begin(struct drm_device *dev,
+ struct drm_connector_list_iter *iter);
struct drm_connector *
drm_connector_list_iter_next(struct drm_connector_list_iter *iter);
-void drm_connector_list_iter_put(struct drm_connector_list_iter *iter);
+void drm_connector_list_iter_end(struct drm_connector_list_iter *iter);
/**
* drm_for_each_connector_iter - connector_list iterator macro
* @iter: &struct drm_connector_list_iter
*
* Note that @connector is only valid within the list body, if you want to use
- * @connector after calling drm_connector_list_iter_put() then you need to grab
- * your own reference first using drm_connector_reference().
+ * @connector after calling drm_connector_list_iter_end() then you need to grab
+ * your own reference first using drm_connector_begin().
*/
#define drm_for_each_connector_iter(connector, iter) \
while ((connector = drm_connector_list_iter_next(iter)))
* Target vertical blank period when a page flip
* should take effect.
*/
-
u32 target_vblank;
+ /**
+ * @pageflip_flags:
+ *
+ * DRM_MODE_PAGE_FLIP_* flags, as passed to the page flip ioctl.
+ * Zero in any other case.
+ */
+ u32 pageflip_flags;
+
/**
* @event:
*
* drm_crtc_arm_vblank_event(). See the documentation of that function
* for a detailed discussion of the constraints it needs to be used
* safely.
+ *
+ * If the device can't notify of flip completion in a race-free way
+ * at all, then the event should be armed just after the page flip is
+ * committed. In the worst case the driver will send the event to
+ * userspace one frame too late. This doesn't allow for a real atomic
+ * update, but it should avoid tearing.
*/
struct drm_pending_vblank_event *event;
*/
void (*atomic_print_state)(struct drm_printer *p,
const struct drm_crtc_state *state);
+
+ /**
+ * @get_vblank_counter:
+ *
+ * Driver callback for fetching a raw hardware vblank counter for the
+ * CRTC. It's meant to be used by new drivers as the replacement of
+ * &drm_driver.get_vblank_counter hook.
+ *
+ * This callback is optional. If a device doesn't have a hardware
+ * counter, the driver can simply leave the hook as NULL. The DRM core
+ * will account for missed vblank events while interrupts where disabled
+ * based on system timestamps.
+ *
+ * Wraparound handling and loss of events due to modesetting is dealt
+ * with in the DRM core code, as long as drivers call
+ * drm_crtc_vblank_off() and drm_crtc_vblank_on() when disabling or
+ * enabling a CRTC.
+ *
+ * Returns:
+ *
+ * Raw vblank counter value.
+ */
+ u32 (*get_vblank_counter)(struct drm_crtc *crtc);
+
+ /**
+ * @enable_vblank:
+ *
+ * Enable vblank interrupts for the CRTC. It's meant to be used by
+ * new drivers as the replacement of &drm_driver.enable_vblank hook.
+ *
+ * Returns:
+ *
+ * Zero on success, appropriate errno if the vblank interrupt cannot
+ * be enabled.
+ */
+ int (*enable_vblank)(struct drm_crtc *crtc);
+
+ /**
+ * @disable_vblank:
+ *
+ * Disable vblank interrupts for the CRTC. It's meant to be used by
+ * new drivers as the replacement of &drm_driver.disable_vblank hook.
+ */
+ void (*disable_vblank)(struct drm_crtc *crtc);
};
/**
* Debugfs directory for this CRTC.
*/
struct dentry *debugfs_entry;
+#endif
/**
* @crc:
* Configuration settings of CRC capture.
*/
struct drm_crtc_crc crc;
-#endif
/**
* @fence_context:
#define DP_TEST_LANE_COUNT 0x220
#define DP_TEST_PATTERN 0x221
+# define DP_NO_TEST_PATTERN 0x0
+# define DP_COLOR_RAMP 0x1
+# define DP_BLACK_AND_WHITE_VERTICAL_LINES 0x2
+# define DP_COLOR_SQUARE 0x3
+
+#define DP_TEST_H_TOTAL_HI 0x222
+#define DP_TEST_H_TOTAL_LO 0x223
+
+#define DP_TEST_V_TOTAL_HI 0x224
+#define DP_TEST_V_TOTAL_LO 0x225
+
+#define DP_TEST_H_START_HI 0x226
+#define DP_TEST_H_START_LO 0x227
+
+#define DP_TEST_V_START_HI 0x228
+#define DP_TEST_V_START_LO 0x229
+
+#define DP_TEST_HSYNC_HI 0x22A
+# define DP_TEST_HSYNC_POLARITY (1 << 7)
+# define DP_TEST_HSYNC_WIDTH_HI_MASK (127 << 0)
+#define DP_TEST_HSYNC_WIDTH_LO 0x22B
+
+#define DP_TEST_VSYNC_HI 0x22C
+# define DP_TEST_VSYNC_POLARITY (1 << 7)
+# define DP_TEST_VSYNC_WIDTH_HI_MASK (127 << 0)
+#define DP_TEST_VSYNC_WIDTH_LO 0x22D
+
+#define DP_TEST_H_WIDTH_HI 0x22E
+#define DP_TEST_H_WIDTH_LO 0x22F
+
+#define DP_TEST_V_HEIGHT_HI 0x230
+#define DP_TEST_V_HEIGHT_LO 0x231
+
+#define DP_TEST_MISC0 0x232
+# define DP_TEST_SYNC_CLOCK (1 << 0)
+# define DP_TEST_COLOR_FORMAT_MASK (3 << 1)
+# define DP_TEST_COLOR_FORMAT_SHIFT 1
+# define DP_COLOR_FORMAT_RGB (0 << 1)
+# define DP_COLOR_FORMAT_YCbCr422 (1 << 1)
+# define DP_COLOR_FORMAT_YCbCr444 (2 << 1)
+# define DP_TEST_DYNAMIC_RANGE_CEA (1 << 3)
+# define DP_TEST_YCBCR_COEFFICIENTS (1 << 4)
+# define DP_YCBCR_COEFFICIENTS_ITU601 (0 << 4)
+# define DP_YCBCR_COEFFICIENTS_ITU709 (1 << 4)
+# define DP_TEST_BIT_DEPTH_MASK (7 << 5)
+# define DP_TEST_BIT_DEPTH_SHIFT 5
+# define DP_TEST_BIT_DEPTH_6 (0 << 5)
+# define DP_TEST_BIT_DEPTH_8 (1 << 5)
+# define DP_TEST_BIT_DEPTH_10 (2 << 5)
+# define DP_TEST_BIT_DEPTH_12 (3 << 5)
+# define DP_TEST_BIT_DEPTH_16 (4 << 5)
+
+#define DP_TEST_MISC1 0x233
+# define DP_TEST_REFRESH_DENOMINATOR (1 << 0)
+# define DP_TEST_INTERLACED (1 << 1)
+
+#define DP_TEST_REFRESH_RATE_NUMERATOR 0x234
#define DP_TEST_CRC_R_CR 0x240
#define DP_TEST_CRC_G_Y 0x242
* @name: user-visible name of this AUX channel and the I2C-over-AUX adapter
* @ddc: I2C adapter that can be used for I2C-over-AUX communication
* @dev: pointer to struct device that is the parent for this AUX channel
+ * @crtc: backpointer to the crtc that is currently using this AUX channel
* @hw_mutex: internal mutex used for locking transfers
+ * @crc_work: worker that captures CRCs for each frame
+ * @crc_count: counter of captured frame CRCs
* @transfer: transfers a message representing a single AUX transaction
*
* The .dev field should be set to a pointer to the device that implements
const char *name;
struct i2c_adapter ddc;
struct device *dev;
+ struct drm_crtc *crtc;
struct mutex hw_mutex;
+ struct work_struct crc_work;
+ u8 crc_count;
ssize_t (*transfer)(struct drm_dp_aux *aux,
struct drm_dp_aux_msg *msg);
/**
int drm_dp_aux_register(struct drm_dp_aux *aux);
void drm_dp_aux_unregister(struct drm_dp_aux *aux);
+int drm_dp_start_crc(struct drm_dp_aux *aux, struct drm_crtc *crtc);
+int drm_dp_stop_crc(struct drm_dp_aux *aux);
+
#endif /* _DRM_DP_HELPER_H_ */
* structure for GEM drivers.
*/
struct drm_driver {
-
/**
* @load:
*
* See drm_dev_init() and drm_dev_register() for proper and
* race-free way to set up a &struct drm_device.
*
+ * This is deprecated, do not use!
+ *
* Returns:
*
* Zero on success, non-zero value on failure.
*/
int (*load) (struct drm_device *, unsigned long flags);
+
+ /**
+ * @open:
+ *
+ * Driver callback when a new &struct drm_file is opened. Useful for
+ * setting up driver-private data structures like buffer allocators,
+ * execution contexts or similar things. Such driver-private resources
+ * must be released again in @postclose.
+ *
+ * Since the display/modeset side of DRM can only be owned by exactly
+ * one &struct drm_file (see &drm_file.is_master and &drm_device.master)
+ * there should never be a need to set up any modeset related resources
+ * in this callback. Doing so would be a driver design bug.
+ *
+ * Returns:
+ *
+ * 0 on success, a negative error code on failure, which will be
+ * promoted to userspace as the result of the open() system call.
+ */
int (*open) (struct drm_device *, struct drm_file *);
+
+ /**
+ * @preclose:
+ *
+ * One of the driver callbacks when a new &struct drm_file is closed.
+ * Useful for tearing down driver-private data structures allocated in
+ * @open like buffer allocators, execution contexts or similar things.
+ *
+ * Since the display/modeset side of DRM can only be owned by exactly
+ * one &struct drm_file (see &drm_file.is_master and &drm_device.master)
+ * there should never be a need to tear down any modeset related
+ * resources in this callback. Doing so would be a driver design bug.
+ *
+ * FIXME: It is not really clear why there's both @preclose and
+ * @postclose. Without a really good reason, use @postclose only.
+ */
void (*preclose) (struct drm_device *, struct drm_file *file_priv);
+
+ /**
+ * @postclose:
+ *
+ * One of the driver callbacks when a new &struct drm_file is closed.
+ * Useful for tearing down driver-private data structures allocated in
+ * @open like buffer allocators, execution contexts or similar things.
+ *
+ * Since the display/modeset side of DRM can only be owned by exactly
+ * one &struct drm_file (see &drm_file.is_master and &drm_device.master)
+ * there should never be a need to tear down any modeset related
+ * resources in this callback. Doing so would be a driver design bug.
+ *
+ * FIXME: It is not really clear why there's both @preclose and
+ * @postclose. Without a really good reason, use @postclose only.
+ */
void (*postclose) (struct drm_device *, struct drm_file *);
+
+ /**
+ * @lastclose:
+ *
+ * Called when the last &struct drm_file has been closed and there's
+ * currently no userspace client for the &struct drm_device.
+ *
+ * Modern drivers should only use this to force-restore the fbdev
+ * framebuffer using drm_fb_helper_restore_fbdev_mode_unlocked().
+ * Anything else would indicate there's something seriously wrong.
+ * Modern drivers can also use this to execute delayed power switching
+ * state changes, e.g. in conjunction with the :ref:`vga_switcheroo`
+ * infrastructure.
+ *
+ * This is called after @preclose and @postclose have been called.
+ *
+ * NOTE:
+ *
+ * All legacy drivers use this callback to de-initialize the hardware.
+ * This is purely because of the shadow-attach model, where the DRM
+ * kernel driver does not really own the hardware. Instead ownershipe is
+ * handled with the help of userspace through an inheritedly racy dance
+ * to set/unset the VT into raw mode.
+ *
+ * Legacy drivers initialize the hardware in the @firstopen callback,
+ * which isn't even called for modern drivers.
+ */
void (*lastclose) (struct drm_device *);
/**
*
* Driver callback for fetching a raw hardware vblank counter for the
* CRTC specified with the pipe argument. If a device doesn't have a
- * hardware counter, the driver can simply use
- * drm_vblank_no_hw_counter() function. The DRM core will account for
- * missed vblank events while interrupts where disabled based on system
- * timestamps.
+ * hardware counter, the driver can simply leave the hook as NULL.
+ * The DRM core will account for missed vblank events while interrupts
+ * where disabled based on system timestamps.
*
* Wraparound handling and loss of events due to modesetting is dealt
* with in the DRM core code, as long as drivers call
* drm_crtc_vblank_off() and drm_crtc_vblank_on() when disabling or
* enabling a CRTC.
*
+ * This is deprecated and should not be used by new drivers.
+ * Use &drm_crtc_funcs.get_vblank_counter instead.
+ *
* Returns:
*
* Raw vblank counter value.
* Enable vblank interrupts for the CRTC specified with the pipe
* argument.
*
+ * This is deprecated and should not be used by new drivers.
+ * Use &drm_crtc_funcs.enable_vblank instead.
+ *
* Returns:
*
* Zero on success, appropriate errno if the given @crtc's vblank
*
* Disable vblank interrupts for the CRTC specified with the pipe
* argument.
+ *
+ * This is deprecated and should not be used by new drivers.
+ * Use &drm_crtc_funcs.disable_vblank instead.
*/
void (*disable_vblank) (struct drm_device *dev, unsigned int pipe);
void (*master_drop)(struct drm_device *dev, struct drm_file *file_priv);
int (*debugfs_init)(struct drm_minor *minor);
- void (*debugfs_cleanup)(struct drm_minor *minor);
/**
* @gem_free_object: deconstructor for drm_gem_objects
const struct drm_display_mode *mode);
#ifdef CONFIG_DRM_LOAD_EDID_FIRMWARE
-int drm_load_edid_firmware(struct drm_connector *connector);
+struct edid *drm_load_edid_firmware(struct drm_connector *connector);
#else
-static inline int drm_load_edid_firmware(struct drm_connector *connector)
+static inline struct edid *
+drm_load_edid_firmware(struct drm_connector *connector)
{
- return 0;
+ return ERR_PTR(-ENOENT);
}
#endif
struct drm_display_mode *drm_mode_find_dmt(struct drm_device *dev,
int hsize, int vsize, int fresh,
bool rb);
-
#endif /* __DRM_EDID_H__ */
.fb_blank = drm_fb_helper_blank, \
.fb_pan_display = drm_fb_helper_pan_display, \
.fb_debug_enter = drm_fb_helper_debug_enter, \
- .fb_debug_leave = drm_fb_helper_debug_leave
+ .fb_debug_leave = drm_fb_helper_debug_leave, \
+ .fb_ioctl = drm_fb_helper_ioctl
#ifdef CONFIG_DRM_FBDEV_EMULATION
void drm_fb_helper_prepare(struct drm_device *dev, struct drm_fb_helper *helper,
struct fb_info *drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper);
void drm_fb_helper_unregister_fbi(struct drm_fb_helper *fb_helper);
-void drm_fb_helper_release_fbi(struct drm_fb_helper *fb_helper);
void drm_fb_helper_fill_var(struct fb_info *info, struct drm_fb_helper *fb_helper,
uint32_t fb_width, uint32_t fb_height);
void drm_fb_helper_fill_fix(struct fb_info *info, uint32_t pitch,
int drm_fb_helper_setcmap(struct fb_cmap *cmap, struct fb_info *info);
+int drm_fb_helper_ioctl(struct fb_info *info, unsigned int cmd,
+ unsigned long arg);
+
int drm_fb_helper_hotplug_event(struct drm_fb_helper *fb_helper);
int drm_fb_helper_initial_config(struct drm_fb_helper *fb_helper, int bpp_sel);
int drm_fb_helper_single_add_all_connectors(struct drm_fb_helper *fb_helper);
static inline void drm_fb_helper_unregister_fbi(struct drm_fb_helper *fb_helper)
{
}
-static inline void drm_fb_helper_release_fbi(struct drm_fb_helper *fb_helper)
-{
-}
static inline void drm_fb_helper_fill_var(struct fb_info *info,
struct drm_fb_helper *fb_helper,
return 0;
}
+static inline int drm_fb_helper_ioctl(struct fb_info *info, unsigned int cmd,
+ unsigned long arg)
+{
+ return 0;
+}
+
static inline void drm_fb_helper_unlink_fbi(struct drm_fb_helper *fb_helper)
{
}
--- /dev/null
+/*
+ * Copyright 1999 Precision Insight, Inc., Cedar Park, Texas.
+ * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California.
+ * Copyright (c) 2009-2010, Code Aurora Forum.
+ * All rights reserved.
+ *
+ * Author: Rickard E. (Rik) Faith <faith@valinux.com>
+ * Author: Gareth Hughes <gareth@valinux.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef _DRM_FILE_H_
+#define _DRM_FILE_H_
+
+#include <linux/types.h>
+#include <linux/completion.h>
+
+#include <uapi/drm/drm.h>
+
+#include <drm/drm_prime.h>
+
+struct dma_fence;
+struct drm_file;
+struct drm_device;
+
+/*
+ * FIXME: Not sure we want to have drm_minor here in the end, but to avoid
+ * header include loops we need it here for now.
+ */
+
+enum drm_minor_type {
+ DRM_MINOR_PRIMARY,
+ DRM_MINOR_CONTROL,
+ DRM_MINOR_RENDER,
+};
+
+/**
+ * struct drm_minor - DRM device minor structure
+ *
+ * This structure represents a DRM minor number for device nodes in /dev.
+ * Entirely opaque to drivers and should never be inspected directly by drivers.
+ * Drivers instead should only interact with &struct drm_file and of course
+ * &struct drm_device, which is also where driver-private data and resources can
+ * be attached to.
+ */
+struct drm_minor {
+ /* private: */
+ int index; /* Minor device number */
+ int type; /* Control or render */
+ struct device *kdev; /* Linux device */
+ struct drm_device *dev;
+
+ struct dentry *debugfs_root;
+
+ struct list_head debugfs_list;
+ struct mutex debugfs_lock; /* Protects debugfs_list. */
+};
+
+/**
+ * struct drm_pending_event - Event queued up for userspace to read
+ *
+ * This represents a DRM event. Drivers can use this as a generic completion
+ * mechanism, which supports kernel-internal &struct completion, &struct dma_fence
+ * and also the DRM-specific &struct drm_event delivery mechanism.
+ */
+struct drm_pending_event {
+ /**
+ * @completion:
+ *
+ * Optional pointer to a kernel internal completion signalled when
+ * drm_send_event() is called, useful to internally synchronize with
+ * nonblocking operations.
+ */
+ struct completion *completion;
+
+ /**
+ * @completion_release:
+ *
+ * Optional callback currently only used by the atomic modeset helpers
+ * to clean up the reference count for the structure @completion is
+ * stored in.
+ */
+ void (*completion_release)(struct completion *completion);
+
+ /**
+ * @event:
+ *
+ * Pointer to the actual event that should be sent to userspace to be
+ * read using drm_read(). Can be optional, since nowadays events are
+ * also used to signal kernel internal threads with @completion or DMA
+ * transactions using @fence.
+ */
+ struct drm_event *event;
+
+ /**
+ * @fence:
+ *
+ * Optional DMA fence to unblock other hardware transactions which
+ * depend upon the nonblocking DRM operation this event represents.
+ */
+ struct dma_fence *fence;
+
+ /**
+ * @file_priv:
+ *
+ * &struct drm_file where @event should be delivered to. Only set when
+ * @event is set.
+ */
+ struct drm_file *file_priv;
+
+ /**
+ * @link:
+ *
+ * Double-linked list to keep track of this event. Can be used by the
+ * driver up to the point when it calls drm_send_event(), after that
+ * this list entry is owned by the core for its own book-keeping.
+ */
+ struct list_head link;
+
+ /**
+ * @pending_link:
+ *
+ * Entry on &drm_file.pending_event_list, to keep track of all pending
+ * events for @file_priv, to allow correct unwinding of them when
+ * userspace closes the file before the event is delivered.
+ */
+ struct list_head pending_link;
+};
+
+/**
+ * struct drm_file - DRM file private data
+ *
+ * This structure tracks DRM state per open file descriptor.
+ */
+struct drm_file {
+ /**
+ * @authenticated:
+ *
+ * Whether the client is allowed to submit rendering, which for legacy
+ * nodes means it must be authenticated.
+ *
+ * See also the :ref:`section on primary nodes and authentication
+ * <drm_primary_node>`.
+ */
+ unsigned authenticated :1;
+
+ /**
+ * @stereo_allowed:
+ *
+ * True when the client has asked us to expose stereo 3D mode flags.
+ */
+ unsigned stereo_allowed :1;
+
+ /**
+ * @universal_planes:
+ *
+ * True if client understands CRTC primary planes and cursor planes
+ * in the plane list. Automatically set when @atomic is set.
+ */
+ unsigned universal_planes:1;
+
+ /** @atomic: True if client understands atomic properties. */
+ unsigned atomic:1;
+
+ /**
+ * @is_master:
+ *
+ * This client is the creator of @master. Protected by struct
+ * &drm_device.master_mutex.
+ *
+ * See also the :ref:`section on primary nodes and authentication
+ * <drm_primary_node>`.
+ */
+ unsigned is_master:1;
+
+ /**
+ * @master:
+ *
+ * Master this node is currently associated with. Only relevant if
+ * drm_is_primary_client() returns true. Note that this only
+ * matches &drm_device.master if the master is the currently active one.
+ *
+ * See also @authentication and @is_master and the :ref:`section on
+ * primary nodes and authentication <drm_primary_node>`.
+ */
+ struct drm_master *master;
+
+ /** @pid: Process that opened this file. */
+ struct pid *pid;
+
+ /** @magic: Authentication magic, see @authenticated. */
+ drm_magic_t magic;
+
+ /**
+ * @lhead:
+ *
+ * List of all open files of a DRM device, linked into
+ * &drm_device.filelist. Protected by &drm_device.filelist_mutex.
+ */
+ struct list_head lhead;
+
+ /** @minor: &struct drm_minor for this file. */
+ struct drm_minor *minor;
+
+ /**
+ * @object_idr:
+ *
+ * Mapping of mm object handles to object pointers. Used by the GEM
+ * subsystem. Protected by @table_lock.
+ */
+ struct idr object_idr;
+
+ /** @table_lock: Protects @object_idr. */
+ spinlock_t table_lock;
+
+ /** @filp: Pointer to the core file structure. */
+ struct file *filp;
+
+ /**
+ * @driver_priv:
+ *
+ * Optional pointer for driver private data. Can be allocated in
+ * &drm_driver.open and should be freed in &drm_driver.postclose.
+ */
+ void *driver_priv;
+
+ /**
+ * @fbs:
+ *
+ * List of &struct drm_framebuffer associated with this file, using the
+ * &drm_framebuffer.filp_head entry.
+ *
+ * Protected by @fbs_lock. Note that the @fbs list holds a reference on
+ * the framebuffer object to prevent it from untimely disappearing.
+ */
+ struct list_head fbs;
+
+ /** @fbs_lock: Protects @fbs. */
+ struct mutex fbs_lock;
+
+ /**
+ * @blobs:
+ *
+ * User-created blob properties; this retains a reference on the
+ * property.
+ *
+ * Protected by @drm_mode_config.blob_lock;
+ */
+ struct list_head blobs;
+
+ /** @event_wait: Waitqueue for new events added to @event_list. */
+ wait_queue_head_t event_wait;
+
+ /**
+ * @pending_event_list:
+ *
+ * List of pending &struct drm_pending_event, used to clean up pending
+ * events in case this file gets closed before the event is signalled.
+ * Uses the &drm_pending_event.pending_link entry.
+ *
+ * Protect by &drm_device.event_lock.
+ */
+ struct list_head pending_event_list;
+
+ /**
+ * @event_list:
+ *
+ * List of &struct drm_pending_event, ready for delivery to userspace
+ * through drm_read(). Uses the &drm_pending_event.link entry.
+ *
+ * Protect by &drm_device.event_lock.
+ */
+ struct list_head event_list;
+
+ /**
+ * @event_space:
+ *
+ * Available event space to prevent userspace from
+ * exhausting kernel memory. Currently limited to the fairly arbitrary
+ * value of 4KB.
+ */
+ int event_space;
+
+ /** @event_read_lock: Serializes drm_read(). */
+ struct mutex event_read_lock;
+
+ /**
+ * @prime:
+ *
+ * Per-file buffer caches used by the PRIME buffer sharing code.
+ */
+ struct drm_prime_file_private prime;
+
+ /* private: */
+ unsigned long lock_count; /* DRI1 legacy lock count */
+};
+
+/**
+ * drm_is_primary_client - is this an open file of the primary node
+ * @file_priv: DRM file
+ *
+ * Returns true if this is an open file of the primary node, i.e.
+ * &drm_file.minor of @file_priv is a primary minor.
+ *
+ * See also the :ref:`section on primary nodes and authentication
+ * <drm_primary_node>`.
+ */
+static inline bool drm_is_primary_client(const struct drm_file *file_priv)
+{
+ return file_priv->minor->type == DRM_MINOR_PRIMARY;
+}
+
+/**
+ * drm_is_render_client - is this an open file of the render node
+ * @file_priv: DRM file
+ *
+ * Returns true if this is an open file of the render node, i.e.
+ * &drm_file.minor of @file_priv is a render minor.
+ *
+ * See also the :ref:`section on render nodes <drm_render_node>`.
+ */
+static inline bool drm_is_render_client(const struct drm_file *file_priv)
+{
+ return file_priv->minor->type == DRM_MINOR_RENDER;
+}
+
+/**
+ * drm_is_control_client - is this an open file of the control node
+ * @file_priv: DRM file
+ *
+ * Control nodes are deprecated and in the process of getting removed from the
+ * DRM userspace API. Do not ever use!
+ */
+static inline bool drm_is_control_client(const struct drm_file *file_priv)
+{
+ return file_priv->minor->type == DRM_MINOR_CONTROL;
+}
+
+int drm_open(struct inode *inode, struct file *filp);
+ssize_t drm_read(struct file *filp, char __user *buffer,
+ size_t count, loff_t *offset);
+int drm_release(struct inode *inode, struct file *filp);
+unsigned int drm_poll(struct file *filp, struct poll_table_struct *wait);
+int drm_event_reserve_init_locked(struct drm_device *dev,
+ struct drm_file *file_priv,
+ struct drm_pending_event *p,
+ struct drm_event *e);
+int drm_event_reserve_init(struct drm_device *dev,
+ struct drm_file *file_priv,
+ struct drm_pending_event *p,
+ struct drm_event *e);
+void drm_event_cancel_free(struct drm_device *dev,
+ struct drm_pending_event *p);
+void drm_send_event_locked(struct drm_device *dev, struct drm_pending_event *e);
+void drm_send_event(struct drm_device *dev, struct drm_pending_event *e);
+
+#endif /* _DRM_FILE_H_ */
* cleanup (like releasing the reference(s) on the backing GEM bo(s))
* should be deferred. In cases like this, the driver would like to
* hold a ref to the fb even though it has already been removed from
- * userspace perspective. See drm_framebuffer_reference() and
- * drm_framebuffer_unreference().
+ * userspace perspective. See drm_framebuffer_get() and
+ * drm_framebuffer_put().
*
* The refcount is stored inside the mode object @base.
*/
void drm_framebuffer_unregister_private(struct drm_framebuffer *fb);
/**
- * drm_framebuffer_reference - incr the fb refcnt
- * @fb: framebuffer
+ * drm_framebuffer_get - acquire a framebuffer reference
+ * @fb: DRM framebuffer
+ *
+ * This function increments the framebuffer's reference count.
+ */
+static inline void drm_framebuffer_get(struct drm_framebuffer *fb)
+{
+ drm_mode_object_get(&fb->base);
+}
+
+/**
+ * drm_framebuffer_put - release a framebuffer reference
+ * @fb: DRM framebuffer
+ *
+ * This function decrements the framebuffer's reference count and frees the
+ * framebuffer if the reference count drops to zero.
+ */
+static inline void drm_framebuffer_put(struct drm_framebuffer *fb)
+{
+ drm_mode_object_put(&fb->base);
+}
+
+/**
+ * drm_framebuffer_reference - acquire a framebuffer reference
+ * @fb: DRM framebuffer
*
- * This functions increments the fb's refcount.
+ * This is a compatibility alias for drm_framebuffer_get() and should not be
+ * used by new code.
*/
static inline void drm_framebuffer_reference(struct drm_framebuffer *fb)
{
- drm_mode_object_reference(&fb->base);
+ drm_framebuffer_get(fb);
}
/**
- * drm_framebuffer_unreference - unref a framebuffer
- * @fb: framebuffer to unref
+ * drm_framebuffer_unreference - release a framebuffer reference
+ * @fb: DRM framebuffer
*
- * This functions decrements the fb's refcount and frees it if it drops to zero.
+ * This is a compatibility alias for drm_framebuffer_put() and should not be
+ * used by new code.
*/
static inline void drm_framebuffer_unreference(struct drm_framebuffer *fb)
{
- drm_mode_object_unreference(&fb->base);
+ drm_framebuffer_put(fb);
}
/**
struct drm_framebuffer *fb)
{
if (fb)
- drm_framebuffer_reference(fb);
+ drm_framebuffer_get(fb);
if (*p)
- drm_framebuffer_unreference(*p);
+ drm_framebuffer_put(*p);
*p = fb;
}
* OTHER DEALINGS IN THE SOFTWARE.
*/
+#include <linux/kref.h>
+
+#include <drm/drm_vma_manager.h>
+
/**
* struct drm_gem_object - GEM buffer object
*
*
* Reference count of this object
*
- * Please use drm_gem_object_reference() to acquire and
- * drm_gem_object_unreference() or drm_gem_object_unreference_unlocked()
- * to release a reference to a GEM buffer object.
+ * Please use drm_gem_object_get() to acquire and drm_gem_object_put()
+ * or drm_gem_object_put_unlocked() to release a reference to a GEM
+ * buffer object.
*/
struct kref refcount;
struct dma_buf_attachment *import_attach;
};
+/**
+ * DEFINE_DRM_GEM_FOPS() - macro to generate file operations for GEM drivers
+ * @name: name for the generated structure
+ *
+ * This macro autogenerates a suitable &struct file_operations for GEM based
+ * drivers, which can be assigned to &drm_driver.fops. Note that this structure
+ * cannot be shared between drivers, because it contains a reference to the
+ * current module using THIS_MODULE.
+ *
+ * Note that the declaration is already marked as static - if you need a
+ * non-static version of this you're probably doing it wrong and will break the
+ * THIS_MODULE reference by accident.
+ */
+#define DEFINE_DRM_GEM_FOPS(name) \
+ static const struct file_operations name = {\
+ .owner = THIS_MODULE,\
+ .open = drm_open,\
+ .release = drm_release,\
+ .unlocked_ioctl = drm_ioctl,\
+ .compat_ioctl = drm_compat_ioctl,\
+ .poll = drm_poll,\
+ .read = drm_read,\
+ .llseek = noop_llseek,\
+ .mmap = drm_gem_mmap,\
+ }
+
void drm_gem_object_release(struct drm_gem_object *obj);
void drm_gem_object_free(struct kref *kref);
int drm_gem_object_init(struct drm_device *dev,
int drm_gem_mmap(struct file *filp, struct vm_area_struct *vma);
/**
- * drm_gem_object_reference - acquire a GEM BO reference
+ * drm_gem_object_get - acquire a GEM buffer object reference
* @obj: GEM buffer object
*
- * This acquires additional reference to @obj. It is illegal to call this
- * without already holding a reference. No locks required.
+ * This function acquires an additional reference to @obj. It is illegal to
+ * call this without already holding a reference. No locks required.
*/
-static inline void
-drm_gem_object_reference(struct drm_gem_object *obj)
+static inline void drm_gem_object_get(struct drm_gem_object *obj)
{
kref_get(&obj->refcount);
}
/**
- * __drm_gem_object_unreference - raw function to release a GEM BO reference
+ * __drm_gem_object_put - raw function to release a GEM buffer object reference
* @obj: GEM buffer object
*
* This function is meant to be used by drivers which are not encumbered with
* &drm_device.struct_mutex legacy locking and which are using the
* gem_free_object_unlocked callback. It avoids all the locking checks and
- * locking overhead of drm_gem_object_unreference() and
- * drm_gem_object_unreference_unlocked().
+ * locking overhead of drm_gem_object_put() and drm_gem_object_put_unlocked().
*
* Drivers should never call this directly in their code. Instead they should
- * wrap it up into a ``driver_gem_object_unreference(struct driver_gem_object
- * *obj)`` wrapper function, and use that. Shared code should never call this, to
+ * wrap it up into a ``driver_gem_object_put(struct driver_gem_object *obj)``
+ * wrapper function, and use that. Shared code should never call this, to
* avoid breaking drivers by accident which still depend upon
* &drm_device.struct_mutex locking.
*/
static inline void
-__drm_gem_object_unreference(struct drm_gem_object *obj)
+__drm_gem_object_put(struct drm_gem_object *obj)
{
kref_put(&obj->refcount, drm_gem_object_free);
}
-void drm_gem_object_unreference_unlocked(struct drm_gem_object *obj);
-void drm_gem_object_unreference(struct drm_gem_object *obj);
+void drm_gem_object_put_unlocked(struct drm_gem_object *obj);
+void drm_gem_object_put(struct drm_gem_object *obj);
+
+/**
+ * drm_gem_object_reference - acquire a GEM buffer object reference
+ * @obj: GEM buffer object
+ *
+ * This is a compatibility alias for drm_gem_object_get() and should not be
+ * used by new code.
+ */
+static inline void drm_gem_object_reference(struct drm_gem_object *obj)
+{
+ drm_gem_object_get(obj);
+}
+
+/**
+ * __drm_gem_object_unreference - raw function to release a GEM buffer object
+ * reference
+ * @obj: GEM buffer object
+ *
+ * This is a compatibility alias for __drm_gem_object_put() and should not be
+ * used by new code.
+ */
+static inline void __drm_gem_object_unreference(struct drm_gem_object *obj)
+{
+ __drm_gem_object_put(obj);
+}
+
+/**
+ * drm_gem_object_unreference_unlocked - release a GEM buffer object reference
+ * @obj: GEM buffer object
+ *
+ * This is a compatibility alias for drm_gem_object_put_unlocked() and should
+ * not be used by new code.
+ */
+static inline void
+drm_gem_object_unreference_unlocked(struct drm_gem_object *obj)
+{
+ drm_gem_object_put_unlocked(obj);
+}
+
+/**
+ * drm_gem_object_unreference - release a GEM buffer object reference
+ * @obj: GEM buffer object
+ *
+ * This is a compatibility alias for drm_gem_object_put() and should not be
+ * used by new code.
+ */
+static inline void drm_gem_object_unreference(struct drm_gem_object *obj)
+{
+ drm_gem_object_put(obj);
+}
int drm_gem_handle_create(struct drm_file *file_priv,
struct drm_gem_object *obj,
return container_of(gem_obj, struct drm_gem_cma_object, base);
}
+/**
+ * DEFINE_DRM_GEM_CMA_FOPS() - macro to generate file operations for CMA drivers
+ * @name: name for the generated structure
+ *
+ * This macro autogenerates a suitable &struct file_operations for CMA based
+ * drivers, which can be assigned to &drm_driver.fops. Note that this structure
+ * cannot be shared between drivers, because it contains a reference to the
+ * current module using THIS_MODULE.
+ *
+ * Note that the declaration is already marked as static - if you need a
+ * non-static version of this you're probably doing it wrong and will break the
+ * THIS_MODULE reference by accident.
+ */
+#define DEFINE_DRM_GEM_CMA_FOPS(name) \
+ static const struct file_operations name = {\
+ .owner = THIS_MODULE,\
+ .open = drm_open,\
+ .release = drm_release,\
+ .unlocked_ioctl = drm_ioctl,\
+ .compat_ioctl = drm_compat_ioctl,\
+ .poll = drm_poll,\
+ .read = drm_read,\
+ .llseek = noop_llseek,\
+ .mmap = drm_gem_cma_mmap,\
+ }
+
/* free GEM object */
void drm_gem_cma_free_object(struct drm_gem_object *gem_obj);
void drm_crtc_vblank_on(struct drm_crtc *crtc);
void drm_vblank_cleanup(struct drm_device *dev);
u32 drm_accurate_vblank_count(struct drm_crtc *crtc);
-u32 drm_vblank_no_hw_counter(struct drm_device *dev, unsigned int pipe);
int drm_calc_vbltimestamp_from_scanoutpos(struct drm_device *dev,
unsigned int pipe, int *max_error,
* but using the internal interval tree to accelerate the search for the
* starting node, and so not safe against removal of elements. It assumes
* that @end is within (or is the upper limit of) the drm_mm allocator.
+ * If [@start, @end] are beyond the range of the drm_mm, the iterator may walk
+ * over the special _unallocated_ &drm_mm.head_node, and may even continue
+ * indefinitely.
*/
#define drm_mm_for_each_node_in_range(node__, mm__, start__, end__) \
for (node__ = __drm_mm_interval_first((mm__), (start__), (end__)-1); \
- node__ && node__->start < (end__); \
+ node__->start < (end__); \
node__ = list_next_entry(node__, node_list))
void drm_mm_scan_init_with_range(struct drm_mm_scan *scan,
* passed-in &drm_atomic_state. This hook is called when the caller
* encountered a &drm_modeset_lock deadlock and needs to drop all
* already acquired locks as part of the deadlock avoidance dance
- * implemented in drm_modeset_lock_backoff().
+ * implemented in drm_modeset_backoff().
*
* Any duplicated state must be invalidated since a concurrent atomic
* update might change it, and the drm atomic interfaces always apply
* itself. Note that the core first calls drm_atomic_state_clear() to
* avoid code duplicate between the clear and free hooks.
*
- * Drivers that implement this must call drm_atomic_state_default_free()
- * to release common resources.
+ * Drivers that implement this must call
+ * drm_atomic_state_default_release() to release common resources.
*/
void (*atomic_state_free)(struct drm_atomic_state *state);
};
* multiple CRTCs.
*/
struct drm_property *tile_property;
+ /**
+ * @link_status_property: Default connector property for link status
+ * of a connector
+ */
+ struct drm_property *link_status_property;
/**
* @plane_type_property: Default plane property to differentiate
* CURSOR, PRIMARY and OVERLAY legacy uses of planes.
/* cursor size */
uint32_t cursor_width, cursor_height;
- struct drm_mode_config_helper_funcs *helper_private;
+ const struct drm_mode_config_helper_funcs *helper_private;
};
void drm_mode_config_init(struct drm_device *dev);
* drm_object_attach_property() before the object is visible to userspace.
*
* - For objects with dynamic lifetimes (as indicated by a non-NULL @free_cb) it
- * provides reference counting through drm_mode_object_reference() and
- * drm_mode_object_unreference(). This is used by &drm_framebuffer,
- * &drm_connector and &drm_property_blob. These objects provide specialized
- * reference counting wrappers.
+ * provides reference counting through drm_mode_object_get() and
+ * drm_mode_object_put(). This is used by &drm_framebuffer, &drm_connector
+ * and &drm_property_blob. These objects provide specialized reference
+ * counting wrappers.
*/
struct drm_mode_object {
uint32_t id;
struct drm_mode_object *drm_mode_object_find(struct drm_device *dev,
uint32_t id, uint32_t type);
-void drm_mode_object_reference(struct drm_mode_object *obj);
-void drm_mode_object_unreference(struct drm_mode_object *obj);
+void drm_mode_object_get(struct drm_mode_object *obj);
+void drm_mode_object_put(struct drm_mode_object *obj);
+
+/**
+ * drm_mode_object_reference - acquire a mode object reference
+ * @obj: DRM mode object
+ *
+ * This is a compatibility alias for drm_mode_object_get() and should not be
+ * used by new code.
+ */
+static inline void drm_mode_object_reference(struct drm_mode_object *obj)
+{
+ drm_mode_object_get(obj);
+}
+
+/**
+ * drm_mode_object_unreference - release a mode object reference
+ * @obj: DRM mode object
+ *
+ * This is a compatibility alias for drm_mode_object_put() and should not be
+ * used by new code.
+ */
+static inline void drm_mode_object_unreference(struct drm_mode_object *obj)
+{
+ drm_mode_object_put(obj);
+}
int drm_object_property_set_value(struct drm_mode_object *obj,
struct drm_property *property,
--- /dev/null
+/*
+ * Internal Header for the Direct Rendering Manager
+ *
+ * Copyright 1999 Precision Insight, Inc., Cedar Park, Texas.
+ * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California.
+ * Copyright (c) 2009-2010, Code Aurora Forum.
+ * All rights reserved.
+ *
+ * Author: Rickard E. (Rik) Faith <faith@valinux.com>
+ * Author: Gareth Hughes <gareth@valinux.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef _DRM_PCI_H_
+#define _DRM_PCI_H_
+
+#include <linux/pci.h>
+
+struct drm_dma_handle;
+struct drm_device;
+struct drm_driver;
+struct drm_master;
+
+extern struct drm_dma_handle *drm_pci_alloc(struct drm_device *dev, size_t size,
+ size_t align);
+extern void drm_pci_free(struct drm_device *dev, struct drm_dma_handle * dmah);
+
+extern int drm_pci_init(struct drm_driver *driver, struct pci_driver *pdriver);
+extern void drm_pci_exit(struct drm_driver *driver, struct pci_driver *pdriver);
+#ifdef CONFIG_PCI
+extern int drm_get_pci_dev(struct pci_dev *pdev,
+ const struct pci_device_id *ent,
+ struct drm_driver *driver);
+extern int drm_pci_set_busid(struct drm_device *dev, struct drm_master *master);
+#else
+static inline int drm_get_pci_dev(struct pci_dev *pdev,
+ const struct pci_device_id *ent,
+ struct drm_driver *driver)
+{
+ return -ENOSYS;
+}
+
+static inline int drm_pci_set_busid(struct drm_device *dev,
+ struct drm_master *master)
+{
+ return -ENOSYS;
+}
+#endif
+
+#define DRM_PCIE_SPEED_25 1
+#define DRM_PCIE_SPEED_50 2
+#define DRM_PCIE_SPEED_80 4
+
+extern int drm_pcie_get_speed_cap_mask(struct drm_device *dev, u32 *speed_mask);
+extern int drm_pcie_get_max_link_width(struct drm_device *dev, u32 *mlw);
+
+#endif /* _DRM_PCI_H_ */
--- /dev/null
+/*
+ * Copyright © 2012 Red Hat
+ * Copyright 1999 Precision Insight, Inc., Cedar Park, Texas.
+ * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California.
+ * Copyright (c) 2009-2010, Code Aurora Forum.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ * Authors:
+ * Dave Airlie <airlied@redhat.com>
+ * Rob Clark <rob.clark@linaro.org>
+ *
+ */
+
+#ifndef __DRM_PRIME_H__
+#define __DRM_PRIME_H__
+
+#include <linux/mutex.h>
+#include <linux/rbtree.h>
+#include <linux/scatterlist.h>
+
+/**
+ * struct drm_prime_file_private - per-file tracking for PRIME
+ *
+ * This just contains the internal &struct dma_buf and handle caches for each
+ * &struct drm_file used by the PRIME core code.
+ */
+
+struct drm_prime_file_private {
+/* private: */
+ struct mutex lock;
+ struct rb_root dmabufs;
+ struct rb_root handles;
+};
+
+struct dma_buf_export_info;
+struct dma_buf;
+
+struct drm_device;
+struct drm_gem_object;
+struct drm_file;
+
+extern struct dma_buf *drm_gem_prime_export(struct drm_device *dev,
+ struct drm_gem_object *obj,
+ int flags);
+extern int drm_gem_prime_handle_to_fd(struct drm_device *dev,
+ struct drm_file *file_priv, uint32_t handle, uint32_t flags,
+ int *prime_fd);
+extern struct drm_gem_object *drm_gem_prime_import(struct drm_device *dev,
+ struct dma_buf *dma_buf);
+extern int drm_gem_prime_fd_to_handle(struct drm_device *dev,
+ struct drm_file *file_priv, int prime_fd, uint32_t *handle);
+struct dma_buf *drm_gem_dmabuf_export(struct drm_device *dev,
+ struct dma_buf_export_info *exp_info);
+extern void drm_gem_dmabuf_release(struct dma_buf *dma_buf);
+
+extern int drm_prime_sg_to_page_addr_arrays(struct sg_table *sgt, struct page **pages,
+ dma_addr_t *addrs, int max_pages);
+extern struct sg_table *drm_prime_pages_to_sg(struct page **pages, unsigned int nr_pages);
+extern void drm_prime_gem_destroy(struct drm_gem_object *obj, struct sg_table *sg);
+
+
+#endif /* __DRM_PRIME_H__ */
#ifndef DRM_PRINT_H_
#define DRM_PRINT_H_
+#include <linux/compiler.h>
+#include <linux/printk.h>
#include <linux/seq_file.h>
#include <linux/device.h>
void __drm_printfn_info(struct drm_printer *p, struct va_format *vaf);
void __drm_printfn_debug(struct drm_printer *p, struct va_format *vaf);
+__printf(2, 3)
void drm_printf(struct drm_printer *p, const char *f, ...);
* Blobs are used to store bigger values than what fits directly into the 64
* bits available for a &drm_property.
*
- * Blobs are reference counted using drm_property_reference_blob() and
- * drm_property_unreference_blob(). They are created using
- * drm_property_create_blob().
+ * Blobs are reference counted using drm_property_blob_get() and
+ * drm_property_blob_put(). They are created using drm_property_create_blob().
*/
struct drm_property_blob {
struct drm_mode_object base;
const void *data,
struct drm_mode_object *obj_holds_id,
struct drm_property *prop_holds_id);
-struct drm_property_blob *drm_property_reference_blob(struct drm_property_blob *blob);
-void drm_property_unreference_blob(struct drm_property_blob *blob);
+struct drm_property_blob *drm_property_blob_get(struct drm_property_blob *blob);
+void drm_property_blob_put(struct drm_property_blob *blob);
+
+/**
+ * drm_property_reference_blob - acquire a blob property reference
+ * @blob: DRM blob property
+ *
+ * This is a compatibility alias for drm_property_blob_get() and should not be
+ * used by new code.
+ */
+static inline struct drm_property_blob *
+drm_property_reference_blob(struct drm_property_blob *blob)
+{
+ return drm_property_blob_get(blob);
+}
+
+/**
+ * drm_property_unreference_blob - release a blob property reference
+ * @blob: DRM blob property
+ *
+ * This is a compatibility alias for drm_property_blob_put() and should not be
+ * used by new code.
+ */
+static inline void
+drm_property_unreference_blob(struct drm_property_blob *blob)
+{
+ drm_property_blob_put(blob);
+}
/**
* drm_connector_find - find property object
--- /dev/null
+/*
+ * Copyright (c) 2015 NVIDIA Corporation. All rights reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sub license,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the
+ * next paragraph) shall be included in all copies or substantial portions
+ * of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef DRM_SCDC_HELPER_H
+#define DRM_SCDC_HELPER_H
+
+#include <linux/i2c.h>
+#include <linux/types.h>
+
+#define SCDC_SINK_VERSION 0x01
+
+#define SCDC_SOURCE_VERSION 0x02
+
+#define SCDC_UPDATE_0 0x10
+#define SCDC_READ_REQUEST_TEST (1 << 2)
+#define SCDC_CED_UPDATE (1 << 1)
+#define SCDC_STATUS_UPDATE (1 << 0)
+
+#define SCDC_UPDATE_1 0x11
+
+#define SCDC_TMDS_CONFIG 0x20
+#define SCDC_TMDS_BIT_CLOCK_RATIO_BY_40 (1 << 1)
+#define SCDC_TMDS_BIT_CLOCK_RATIO_BY_10 (0 << 1)
+#define SCDC_SCRAMBLING_ENABLE (1 << 0)
+
+#define SCDC_SCRAMBLER_STATUS 0x21
+#define SCDC_SCRAMBLING_STATUS (1 << 0)
+
+#define SCDC_CONFIG_0 0x30
+#define SCDC_READ_REQUEST_ENABLE (1 << 0)
+
+#define SCDC_STATUS_FLAGS_0 0x40
+#define SCDC_CH2_LOCK (1 < 3)
+#define SCDC_CH1_LOCK (1 < 2)
+#define SCDC_CH0_LOCK (1 < 1)
+#define SCDC_CH_LOCK_MASK (SCDC_CH2_LOCK | SCDC_CH1_LOCK | SCDC_CH0_LOCK)
+#define SCDC_CLOCK_DETECT (1 << 0)
+
+#define SCDC_STATUS_FLAGS_1 0x41
+
+#define SCDC_ERR_DET_0_L 0x50
+#define SCDC_ERR_DET_0_H 0x51
+#define SCDC_ERR_DET_1_L 0x52
+#define SCDC_ERR_DET_1_H 0x53
+#define SCDC_ERR_DET_2_L 0x54
+#define SCDC_ERR_DET_2_H 0x55
+#define SCDC_CHANNEL_VALID (1 << 7)
+
+#define SCDC_ERR_DET_CHECKSUM 0x56
+
+#define SCDC_TEST_CONFIG_0 0xc0
+#define SCDC_TEST_READ_REQUEST (1 << 7)
+#define SCDC_TEST_READ_REQUEST_DELAY(x) ((x) & 0x7f)
+
+#define SCDC_MANUFACTURER_IEEE_OUI 0xd0
+#define SCDC_MANUFACTURER_IEEE_OUI_SIZE 3
+
+#define SCDC_DEVICE_ID 0xd3
+#define SCDC_DEVICE_ID_SIZE 8
+
+#define SCDC_DEVICE_HARDWARE_REVISION 0xdb
+#define SCDC_GET_DEVICE_HARDWARE_REVISION_MAJOR(x) (((x) >> 4) & 0xf)
+#define SCDC_GET_DEVICE_HARDWARE_REVISION_MINOR(x) (((x) >> 0) & 0xf)
+
+#define SCDC_DEVICE_SOFTWARE_MAJOR_REVISION 0xdc
+#define SCDC_DEVICE_SOFTWARE_MINOR_REVISION 0xdd
+
+#define SCDC_MANUFACTURER_SPECIFIC 0xde
+#define SCDC_MANUFACTURER_SPECIFIC_SIZE 34
+
+ssize_t drm_scdc_read(struct i2c_adapter *adapter, u8 offset, void *buffer,
+ size_t size);
+ssize_t drm_scdc_write(struct i2c_adapter *adapter, u8 offset,
+ const void *buffer, size_t size);
+
+/**
+ * drm_scdc_readb - read a single byte from SCDC
+ * @adapter: I2C adapter
+ * @offset: offset of register to read
+ * @value: return location for the register value
+ *
+ * Reads a single byte from SCDC. This is a convenience wrapper around the
+ * drm_scdc_read() function.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+static inline int drm_scdc_readb(struct i2c_adapter *adapter, u8 offset,
+ u8 *value)
+{
+ return drm_scdc_read(adapter, offset, value, sizeof(*value));
+}
+
+/**
+ * drm_scdc_writeb - write a single byte to SCDC
+ * @adapter: I2C adapter
+ * @offset: offset of register to read
+ * @value: return location for the register value
+ *
+ * Writes a single byte to SCDC. This is a convenience wrapper around the
+ * drm_scdc_write() function.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+static inline int drm_scdc_writeb(struct i2c_adapter *adapter, u8 offset,
+ u8 value)
+{
+ return drm_scdc_write(adapter, offset, &value, sizeof(value));
+}
+
+/**
+ * drm_scdc_set_scrambling - enable scrambling
+ * @adapter: I2C adapter for DDC channel
+ * @enable: bool to indicate if scrambling is to be enabled/disabled
+ *
+ * Writes the TMDS config register over SCDC channel, and:
+ * enables scrambling when enable = 1
+ * disables scrambling when enable = 0
+ *
+ * Returns:
+ * True if scrambling is set/reset successfully, false otherwise.
+ */
+bool drm_scdc_set_scrambling(struct i2c_adapter *adapter, bool enable);
+
+/**
+ * drm_scdc_set_high_tmds_clock_ratio - set TMDS clock ratio
+ * @adapter: I2C adapter for DDC channel
+ * @set: ret or reset the high clock ratio
+ *
+ * Writes to the TMDS config register over SCDC channel, and:
+ * sets TMDS clock ratio to 1/40 when set = 1
+ * sets TMDS clock ratio to 1/10 when set = 0
+ *
+ * Returns:
+ * True if write is successful, false otherwise.
+ */
+bool drm_scdc_set_high_tmds_clock_ratio(struct i2c_adapter *adapter, bool set);
+#endif
#include <drm/drm_mm.h>
#include <linux/mm.h>
-#include <linux/module.h>
#include <linux/rbtree.h>
#include <linux/spinlock.h>
#include <linux/types.h>
0x030000, 0xff0000, \
(unsigned long) info }
+#define INTEL_I810_IDS(info) \
+ INTEL_VGA_DEVICE(0x7121, info), /* I810 */ \
+ INTEL_VGA_DEVICE(0x7123, info), /* I810_DC100 */ \
+ INTEL_VGA_DEVICE(0x7125, info) /* I810_E */
+
+#define INTEL_I815_IDS(info) \
+ INTEL_VGA_DEVICE(0x1132, info) /* I815*/
+
#define INTEL_I830_IDS(info) \
INTEL_VGA_DEVICE(0x3577, info)
INTEL_VGA_DEVICE(0x1923, info), /* ULT GT3 */ \
INTEL_VGA_DEVICE(0x1926, info), /* ULT GT3 */ \
INTEL_VGA_DEVICE(0x1927, info), /* ULT GT3 */ \
- INTEL_VGA_DEVICE(0x192B, info) /* Halo GT3 */ \
+ INTEL_VGA_DEVICE(0x192B, info), /* Halo GT3 */ \
+ INTEL_VGA_DEVICE(0x192D, info) /* SRV GT3 */
#define INTEL_SKL_GT4_IDS(info) \
INTEL_VGA_DEVICE(0x1932, info), /* DT GT4 */ \
u64 context, unsigned seqno,
bool signal_on_any);
+bool dma_fence_match_context(struct dma_fence *fence, u64 context);
+
#endif /* __LINUX_DMA_FENCE_ARRAY_H */
};
#define HDMI_IEEE_OUI 0x000c03
+#define HDMI_FORUM_IEEE_OUI 0xc45dd8
#define HDMI_INFOFRAME_HEADER_SIZE 4
#define HDMI_AVI_INFOFRAME_SIZE 13
#define HDMI_SPD_INFOFRAME_SIZE 25
const struct of_dev_auxdata *lookup,
struct device *parent);
extern void of_platform_depopulate(struct device *parent);
+
+extern int devm_of_platform_populate(struct device *dev);
+
+extern void devm_of_platform_depopulate(struct device *dev);
#else
static inline int of_platform_populate(struct device_node *root,
const struct of_device_id *matches,
return -ENODEV;
}
static inline void of_platform_depopulate(struct device *parent) { }
+
+static inline int devm_of_platform_populate(struct device *dev)
+{
+ return -ENODEV;
+}
+
+static inline void devm_of_platform_depopulate(struct device *dev) { }
#endif
#if defined(CONFIG_OF_DYNAMIC) && defined(CONFIG_OF_ADDRESS)
return ww_mutex_lock(&obj->lock, ctx);
}
+/**
+ * reservation_object_trylock - trylock the reservation object
+ * @obj: the reservation object
+ *
+ * Tries to lock the reservation object for exclusive access and modification.
+ * Note, that the lock is only against other writers, readers will run
+ * concurrently with a writer under RCU. The seqlock is used to notify readers
+ * if they overlap with a writer.
+ *
+ * Also note that since no context is provided, no deadlock protection is
+ * possible.
+ *
+ * Returns true if the lock was acquired, false otherwise.
+ */
+static inline bool __must_check
+reservation_object_trylock(struct reservation_object *obj)
+{
+ return ww_mutex_trylock(&obj->lock);
+}
+
/**
* reservation_object_unlock - unlock the reservation object
* @obj: the reservation object
#define DRM_FORMAT_AYUV fourcc_code('A', 'Y', 'U', 'V') /* [31:0] A:Y:Cb:Cr 8:8:8:8 little endian */
+/*
+ * 2 plane RGB + A
+ * index 0 = RGB plane, same format as the corresponding non _A8 format has
+ * index 1 = A plane, [7:0] A
+ */
+#define DRM_FORMAT_XRGB8888_A8 fourcc_code('X', 'R', 'A', '8')
+#define DRM_FORMAT_XBGR8888_A8 fourcc_code('X', 'B', 'A', '8')
+#define DRM_FORMAT_RGBX8888_A8 fourcc_code('R', 'X', 'A', '8')
+#define DRM_FORMAT_BGRX8888_A8 fourcc_code('B', 'X', 'A', '8')
+#define DRM_FORMAT_RGB888_A8 fourcc_code('R', '8', 'A', '8')
+#define DRM_FORMAT_BGR888_A8 fourcc_code('B', '8', 'A', '8')
+#define DRM_FORMAT_RGB565_A8 fourcc_code('R', '5', 'A', '8')
+#define DRM_FORMAT_BGR565_A8 fourcc_code('B', '5', 'A', '8')
+
/*
* 2 plane YCbCr
* index 0 = Y plane, [7:0] Y
#define DRM_MODE_DIRTY_ON 1
#define DRM_MODE_DIRTY_ANNOTATE 2
+/* Link Status options */
+#define DRM_MODE_LINK_STATUS_GOOD 0
+#define DRM_MODE_LINK_STATUS_BAD 1
+
struct drm_mode_modeinfo {
__u32 clock;
__u16 hdisplay;
#define DRM_I915_OVERLAY_PUT_IMAGE 0x27
#define DRM_I915_OVERLAY_ATTRS 0x28
#define DRM_I915_GEM_EXECBUFFER2 0x29
+#define DRM_I915_GEM_EXECBUFFER2_WR DRM_I915_GEM_EXECBUFFER2
#define DRM_I915_GET_SPRITE_COLORKEY 0x2a
#define DRM_I915_SET_SPRITE_COLORKEY 0x2b
#define DRM_I915_GEM_WAIT 0x2c
#define DRM_IOCTL_I915_GEM_INIT DRM_IOW(DRM_COMMAND_BASE + DRM_I915_GEM_INIT, struct drm_i915_gem_init)
#define DRM_IOCTL_I915_GEM_EXECBUFFER DRM_IOW(DRM_COMMAND_BASE + DRM_I915_GEM_EXECBUFFER, struct drm_i915_gem_execbuffer)
#define DRM_IOCTL_I915_GEM_EXECBUFFER2 DRM_IOW(DRM_COMMAND_BASE + DRM_I915_GEM_EXECBUFFER2, struct drm_i915_gem_execbuffer2)
+#define DRM_IOCTL_I915_GEM_EXECBUFFER2_WR DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_GEM_EXECBUFFER2_WR, struct drm_i915_gem_execbuffer2)
#define DRM_IOCTL_I915_GEM_PIN DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_GEM_PIN, struct drm_i915_gem_pin)
#define DRM_IOCTL_I915_GEM_UNPIN DRM_IOW(DRM_COMMAND_BASE + DRM_I915_GEM_UNPIN, struct drm_i915_gem_unpin)
#define DRM_IOCTL_I915_GEM_BUSY DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_GEM_BUSY, struct drm_i915_gem_busy)
#define I915_PARAM_HAS_SCHEDULER 41
#define I915_PARAM_HUC_STATUS 42
+/* Query whether DRM_I915_GEM_EXECBUFFER2 supports the ability to opt-out of
+ * synchronisation with implicit fencing on individual objects.
+ * See EXEC_OBJECT_ASYNC.
+ */
+#define I915_PARAM_HAS_EXEC_ASYNC 43
+
+/* Query whether DRM_I915_GEM_EXECBUFFER2 supports explicit fence support -
+ * both being able to pass in a sync_file fd to wait upon before executing,
+ * and being able to return a new sync_file fd that is signaled when the
+ * current request is complete. See I915_EXEC_FENCE_IN and I915_EXEC_FENCE_OUT.
+ */
+#define I915_PARAM_HAS_EXEC_FENCE 44
+
typedef struct drm_i915_getparam {
__s32 param;
/*
#define EXEC_OBJECT_SUPPORTS_48B_ADDRESS (1<<3)
#define EXEC_OBJECT_PINNED (1<<4)
#define EXEC_OBJECT_PAD_TO_SIZE (1<<5)
+/* The kernel implicitly tracks GPU activity on all GEM objects, and
+ * synchronises operations with outstanding rendering. This includes
+ * rendering on other devices if exported via dma-buf. However, sometimes
+ * this tracking is too coarse and the user knows better. For example,
+ * if the object is split into non-overlapping ranges shared between different
+ * clients or engines (i.e. suballocating objects), the implicit tracking
+ * by kernel assumes that each operation affects the whole object rather
+ * than an individual range, causing needless synchronisation between clients.
+ * The kernel will also forgo any CPU cache flushes prior to rendering from
+ * the object as the client is expected to be also handling such domain
+ * tracking.
+ *
+ * The kernel maintains the implicit tracking in order to manage resources
+ * used by the GPU - this flag only disables the synchronisation prior to
+ * rendering with this object in this execbuf.
+ *
+ * Opting out of implicit synhronisation requires the user to do its own
+ * explicit tracking to avoid rendering corruption. See, for example,
+ * I915_PARAM_HAS_EXEC_FENCE to order execbufs and execute them asynchronously.
+ */
+#define EXEC_OBJECT_ASYNC (1<<6)
/* All remaining bits are MBZ and RESERVED FOR FUTURE USE */
-#define __EXEC_OBJECT_UNKNOWN_FLAGS -(EXEC_OBJECT_PAD_TO_SIZE<<1)
+#define __EXEC_OBJECT_UNKNOWN_FLAGS -(EXEC_OBJECT_ASYNC<<1)
__u64 flags;
union {
*/
#define I915_EXEC_RESOURCE_STREAMER (1<<15)
-#define __I915_EXEC_UNKNOWN_FLAGS -(I915_EXEC_RESOURCE_STREAMER<<1)
+/* Setting I915_EXEC_FENCE_IN implies that lower_32_bits(rsvd2) represent
+ * a sync_file fd to wait upon (in a nonblocking manner) prior to executing
+ * the batch.
+ *
+ * Returns -EINVAL if the sync_file fd cannot be found.
+ */
+#define I915_EXEC_FENCE_IN (1<<16)
+
+/* Setting I915_EXEC_FENCE_OUT causes the ioctl to return a sync_file fd
+ * in the upper_32_bits(rsvd2) upon success. Ownership of the fd is given
+ * to the caller, and it should be close() after use. (The fd is a regular
+ * file descriptor and will be cleaned up on process termination. It holds
+ * a reference to the request, but nothing else.)
+ *
+ * The sync_file fd can be combined with other sync_file and passed either
+ * to execbuf using I915_EXEC_FENCE_IN, to atomic KMS ioctls (so that a flip
+ * will only occur after this request completes), or to other devices.
+ *
+ * Using I915_EXEC_FENCE_OUT requires use of
+ * DRM_IOCTL_I915_GEM_EXECBUFFER2_WR ioctl so that the result is written
+ * back to userspace. Failure to do so will cause the out-fence to always
+ * be reported as zero, and the real fence fd to be leaked.
+ */
+#define I915_EXEC_FENCE_OUT (1<<17)
+
+#define __I915_EXEC_UNKNOWN_FLAGS (-(I915_EXEC_FENCE_OUT<<1))
#define I915_EXEC_CONTEXT_ID_MASK (0xffffffff)
#define i915_execbuffer2_set_context_id(eb2, context) \
#define IPUV3_CHANNEL_MEM_BG_ASYNC_ALPHA 52
#define IPUV3_NUM_CHANNELS 64
+static inline int ipu_channel_alpha_channel(int ch_num)
+{
+ switch (ch_num) {
+ case IPUV3_CHANNEL_G_MEM_IC_PRP_VF:
+ return IPUV3_CHANNEL_G_MEM_IC_PRP_VF_ALPHA;
+ case IPUV3_CHANNEL_G_MEM_IC_PP:
+ return IPUV3_CHANNEL_G_MEM_IC_PP_ALPHA;
+ case IPUV3_CHANNEL_MEM_FG_SYNC:
+ return IPUV3_CHANNEL_MEM_FG_SYNC_ALPHA;
+ case IPUV3_CHANNEL_MEM_FG_ASYNC:
+ return IPUV3_CHANNEL_MEM_FG_ASYNC_ALPHA;
+ case IPUV3_CHANNEL_MEM_BG_SYNC:
+ return IPUV3_CHANNEL_MEM_BG_SYNC_ALPHA;
+ case IPUV3_CHANNEL_MEM_BG_ASYNC:
+ return IPUV3_CHANNEL_MEM_BG_ASYNC_ALPHA;
+ case IPUV3_CHANNEL_MEM_VDI_PLANE1_COMB:
+ return IPUV3_CHANNEL_MEM_VDI_PLANE1_COMB_ALPHA;
+ default:
+ return -EINVAL;
+ }
+}
+
int ipu_map_irq(struct ipu_soc *ipu, int irq);
int ipu_idmac_channel_irq(struct ipu_soc *ipu, struct ipuv3_channel *channel,
enum ipu_channel_irq irq);
void ipu_dp_put(struct ipu_dp *);
int ipu_dp_enable(struct ipu_soc *ipu);
int ipu_dp_enable_channel(struct ipu_dp *dp);
-void ipu_dp_disable_channel(struct ipu_dp *dp);
+void ipu_dp_disable_channel(struct ipu_dp *dp, bool sync);
void ipu_dp_disable(struct ipu_soc *ipu);
int ipu_dp_setup_channel(struct ipu_dp *dp,
enum ipu_color_space in, enum ipu_color_space out);
int ipu_dp_set_global_alpha(struct ipu_dp *dp, bool enable, u8 alpha,
bool bg_chan);
+/*
+ * IPU Prefetch Resolve Gasket (prg) functions
+ */
+int ipu_prg_max_active_channels(void);
+bool ipu_prg_present(struct ipu_soc *ipu);
+bool ipu_prg_format_supported(struct ipu_soc *ipu, uint32_t format,
+ uint64_t modifier);
+int ipu_prg_enable(struct ipu_soc *ipu);
+void ipu_prg_disable(struct ipu_soc *ipu);
+void ipu_prg_channel_disable(struct ipuv3_channel *ipu_chan);
+int ipu_prg_channel_configure(struct ipuv3_channel *ipu_chan,
+ unsigned int axi_id, unsigned int width,
+ unsigned int height, unsigned int stride,
+ u32 format, unsigned long *eba);
+
/*
* IPU CMOS Sensor Interface (csi) functions
*/
{
current->lockdep_reclaim_gfp = gfp_mask;
}
+EXPORT_SYMBOL_GPL(lockdep_set_current_reclaim_state);
void lockdep_clear_current_reclaim_state(void)
{
current->lockdep_reclaim_gfp = 0;
}
+EXPORT_SYMBOL_GPL(lockdep_clear_current_reclaim_state);
#ifdef CONFIG_LOCK_STAT
static int
--- /dev/null
+///
+/// Use drm_*_get() and drm_*_put() helpers instead of drm_*_reference() and
+/// drm_*_unreference() helpers.
+///
+// Confidence: High
+// Copyright: (C) 2017 NVIDIA Corporation
+// Options: --no-includes --include-headers
+//
+
+virtual patch
+virtual report
+
+@depends on patch@
+expression object;
+@@
+
+(
+- drm_mode_object_reference(object)
++ drm_mode_object_get(object)
+|
+- drm_mode_object_unreference(object)
++ drm_mode_object_put(object)
+|
+- drm_connector_reference(object)
++ drm_connector_get(object)
+|
+- drm_connector_unreference(object)
++ drm_connector_put(object)
+|
+- drm_framebuffer_reference(object)
++ drm_framebuffer_get(object)
+|
+- drm_framebuffer_unreference(object)
++ drm_framebuffer_put(object)
+|
+- drm_gem_object_reference(object)
++ drm_gem_object_get(object)
+|
+- drm_gem_object_unreference(object)
++ drm_gem_object_put(object)
+|
+- __drm_gem_object_unreference(object)
++ __drm_gem_object_put(object)
+|
+- drm_gem_object_unreference_unlocked(object)
++ drm_gem_object_put_unlocked(object)
+|
+- drm_property_reference_blob(object)
++ drm_property_blob_get(object)
+|
+- drm_property_unreference_blob(object)
++ drm_property_blob_put(object)
+)
+
+@r depends on report@
+expression object;
+position p;
+@@
+
+(
+drm_mode_object_unreference@p(object)
+|
+drm_mode_object_reference@p(object)
+|
+drm_connector_unreference@p(object)
+|
+drm_connector_reference@p(object)
+|
+drm_framebuffer_unreference@p(object)
+|
+drm_framebuffer_reference@p(object)
+|
+drm_gem_object_unreference@p(object)
+|
+drm_gem_object_reference@p(object)
+|
+__drm_gem_object_unreference(object)
+|
+drm_gem_object_unreference_unlocked(object)
+|
+drm_property_unreference_blob@p(object)
+|
+drm_property_reference_blob@p(object)
+)
+
+@script:python depends on report@
+object << r.object;
+p << r.p;
+@@
+
+msg="WARNING: use get/put helpers to reference and dereference %s" % (object)
+coccilib.report.print_report(p[0], msg)
fi
if /sbin/modprobe -q i915 mock_selftests=-1; then
+ /sbin/modprobe -q -r i915
echo "drivers/gpu/i915: ok"
else
echo "drivers/gpu/i915: [FAIL]"
projects / karo-tx-linux.git / commitdiff
