mptcp/openmptcprouter
1
0
Fork 0
mirror of https://github.com/Ysurac/openmptcprouter.git synced 2025-03-09 15:40:20 +00:00
Code Issues Releases Wiki Activity
openmptcprouter/6.12/target/linux/bcm27xx/patches-6.12/950-0215-PCI-brcmstb-Add-BCM2712-support.patch
Ycarus (Yannick Chabanois) bdb9b0046f Add bcm27xx 6.12 test support
2024-12-20 14:17:26 +01:00

1235 lines
43 KiB
Diff
Raw Blame History

From cbae17fd95c01508f740a29eca2f6461e3d79885 Mon Sep 17 00:00:00 2001
From: Phil Elwell <phil@raspberrypi.com>
Date: Thu, 11 Jun 2020 09:57:03 +0100
Subject: [PATCH 215/697] PCI: brcmstb: Add BCM2712 support
PCI: brcmstb: Add DT property to control L1SS
The BRCM PCIe block has controls to enable control of the CLKREQ#
signal by the L1SS, and to gate the refclk with the CLKREQ# input.
These controls are mutually exclusive - the upstream code sets the
latter, but some use cases require the former.
Add a Device Tree property - brcm,enable-l1ss - to switch to the
L1SS configuration.
Signed-off-by: Phil Elwell <phil@raspberrypi.com>
PCI: brcmstb: Advertise MSI-X support
Although the BRCMSTB PCIe interface doesn't technically support the
MSI-X spec, in practise it seems to work provided no more than 32
MSI-Xs are required. Add the required flag to the driver to allow
experimentation with devices that demand MSI-X support.
Signed-off-by: Phil Elwell <phil@raspberrypi.com>
PCI: brcmstb: differing register offsets on 2712
pcie-brcmstb: Add 2712 bridge reset support
pcie: 2712 PORT_MASK and rescal support
pcie-brcmstb: don't alter the L1SS debug register
For reasons unknown, this disables the reference clock
pcie-brcmstb: fix BAR2 enable and window decode
Set UBUS ACCESS_EN to let inbound DMA work. Also BCM2712 has grown
an index in the inbound window size decode register.
PCIe: brcmstb: Enable support for 64 MSI-Xs
Signed-off-by: Phil Elwell <phil@raspberrypi.com>
pcie-brcmstb: Suppress read error responses
If the link is down or the EP fails to return a read completion, the
RC's default behaviour is to return an AXI error. This causes fatal
exceptions on A76, so it's better to respond with all 1s instead.
pcie-brcmstb: increase UBUS timeout to cater for link retrain events
pcie-brcmstb: Handle additional inbound regions
Signed-off-by: Phil Elwell <phil@raspberrypi.com>
pcie-brcmstb: Add support for external MSI controller
pcie-brcmstb: add a reasonable default traffic class to priority map
BCM2712 supports multiple traffic classes (TCs) with independent
maximally sized transfer queues for each TC. Traffic classes have no
transaction ordering requirements between them, which facilitates
out-of-order completions and arbitration between posted writes for
data streams that have no dependence on each other.
In addition to the above benefits of splitting endpoint traffic into
individual queues, priorities can be assigned to traffic classes by
a heuristic or deterministic mechanism. The heuristic elevates AXI
QOS priority in accordance with the number of pending transfers in
each TC's queue, but for true priority signalling a forwarding
mechanism using vendor-defined messages is implemented.
Receipt of a 3 DWORD VDM assigns a priority tag to a TC on-the-fly,
and this tag corresponds to a configurable AXI QOS value.
As a simple baseline, assign a linear map of AXI QOS to each tag.
pcie: brcmstb: set up the VDM forwarding interface when setting up QoS
pcie-brcmstb: clean up debug messages
pcie-brcmstb: fix BCM2712A0 PHY PM errata
The power management clock is 54MHz not 50MHz, so adjust the PM clock period
to suit. Powering off the PHY PLL in L1.2 is unsafe, so force it on.
pcie-brcmstb: set CLKREQ functionality according to link partner support
The RC supports either L1 with clock PM or L1 sub-state control, not both
at the same time. Examine the link partner's capabilities to determine
which is the most suitable scheme to use.
pcie: brcmstb: don't reset block bridges in suspend or removal cases
BCM2712 has a single rescal block for all three root complexes, and
holding PCIE1's bridge in reset will hang the chip if a different
RC wants to access any of the rescal registers.
pcie: brcmstb: guard 2712-specific setup with a RC type check
BCM2711 doesn't implement the UBUS control registers.
pcie: brcmstb: On 2712 keeping the PLL powered in L1.x is not required
A separate misconfiguration when enabling SSC (the MDIO registers no
longer do the same thing on BCM2712) had the side-effect of breaking
PLL powerdown and resume sequencing.
Allow entry into a true L1.2 state where analogue is depowered.
pcie: brcmstb: Fix reset warning on probe failure
Signed-off-by: Phil Elwell <phil@raspberrypi.com>
bcm2712: pcie: adjust PHY PLL setup to use a 54MHz input refclk
Use canned MDIO writes from Broadcom that switch the ref_clk output
pair to run from the internal fractional PLL, and set the internal PLL
to expect a 54MHz input reference clock.
Gen3 operation is not guaranteed to be stable in this setup, so default
to gen2.
This only works if the LCPLL is bypassed (requires latest bootloader).
pcie: brcmstb: add missing register writes
drivers: pcie: brcmstb: cater for BCM2712C0 bug dropping QoS on the floor
The AXI QoS value extracted from the request fifo ends up as zero forever.
Disabling this means that "panic" signalling doesn't do anything useful,
but static priorites do work.
Also align the selected TC:QoS map with RP1's expectations of service.
Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
drivers: pcie: brcmstb: shuffle TC priorities up to 8
Use the range 8-11 which puts the highest below HVS but leaves space
below for other 2712 masters.
Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
drivers: pcie: brcmstb: optionally enable QoS features by DT for BCM2712
It's a bad idea to universally enable "realtime" priorities for TCs
across all the RC instances on the chip. Endpoints other than RP1 may
make use of these, so you don't want e.g. NVMe descriptor fetches getting
higher priority than your remote display.
Add two optional DT properties controlling the behaviour - FIFO-based
backpressure QoS or "message-based". Message-based signalling is
fundamentally broken due to a chip bug, so it collapses into a set of
static assignments that RP1 needs.
The default if neither property is specified is to assign everything a
QoS of 0.
Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
drivers: pcie: brcmstb: adjust completion timeouts for bcm2712
Setting the RC config retry timeout makes CRS auto-polling work, but
the UBUS timeout will override the config retry. Both need to be large.
Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
PCI: brcmstb: Change RCB_{MPS,64B}_MODE bits
Upstream commit [1] unconditionally sets the RCB_MPS and RCB_64B bits
that govern where packets are split. We think this is potentially
harmful, particularly on CM4 and Pi 5 where potentially any PCIe devices
could be attached.
Make RCB_MPS conditional on a DT property and never set RCB_64B.
[1] commit 602fb860945f ("PCI: brcmstb: Set RCB_{MPS,64B}_MODE bits")
Signed-off-by: Phil Elwell <phil@raspberrypi.com>
drivers: pci: brcmstb: optionally extend Tperst_clk time during link-up
The RC has a feature that allows for manual control over the deassertion
of the PERST# output pin, which allows the time between refclk active
and reset deassert at the EP to be increased.
Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
PCI: brcmstb: fix broken brcm_pcie_mdio_write() polling
MDIO_WR_DONE() tests bit 31, which is always 0 (==done) as
readw_poll_timeout_atomic does a 16-bit read. Replace with the readl
variant.
Fixes: ca5dcc76314d ("PCI: brcmstb: Replace status loops with read_poll_timeout_atomic()")
Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
PCI: brcmstb: resolve conflicts with upstream CLKREQ# selection
This partially reverts commit 177baa7aeb3f ("Revert "PCI: brcmstb:
Configure HW CLKREQ# mode appropriate for downstream device"")
commit e2596dcf1e9d ("PCI: brcmstb: Configure HW CLKREQ# mode appropriate
for downstream device") upstream.
The upstream implementation should replace the downstream one.
Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
PCI: brcmstb: don't set bridge timeouts on BCM2712
BCM2712 has a slightly different architecture to most other STB chips.
Timeout values are internal to the core and are set in PCIE_MISC not
PCIE_RGR1.
Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
PCI: brcmstb: Enable CRS software visibility after linkup
It appears that bits in the Root Control Register are reset with
perst_n, which means the PCI layer's call to enable CRS prior to
adding/scanning the bus has no effect. Open-code the enable in
brcm_pcie_start_link as a workaround.
Without CRS visibility, configuration reads issued by the CPU don't
retire if the endpoint returns a CRS response - the RC will poll until a
(large) timeout is reached. This means the core can stall for a long
time during boot.
Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
PCI: brcmstb: Set new flags to avoid QOS "holes" on BCM2712D0
Set some flags present (and recommended) in 2712D0, but missing
(and harmless) in 2712C1. In particular, EN_QOS_UPDATE_TIMING_FIX
must be set to avoid spurious QOS=0 when the queue changes from
empty to non-empty, to make D0 match the existing C1 behaviour.
Not enabling "QOS forwarding", which still seems not to help.
Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
drivers: pcie-brcmstb: add best-effort workaround for QoS bug on bcm2712
If a set of read requests are issued by an endpoint, they are streamed
into a resynchronisation FIFO prior to exiting the RC. This FIFO has an
edge case where it can drop QoS for a request to 0 if there's a single
outstanding read request in the FIFO, and another is pushed when the
FIFO is popped. Requests with a QoS of 0 can take hundreds of
microseconds to complete.
By adding an experimentally-determined amount of backpressure on the pop
side, the critical level transition can largely be avoided.
Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
---
drivers/pci/controller/pcie-brcmstb.c | 660 ++++++++++++++++++++++++--
1 file changed, 629 insertions(+), 31 deletions(-)
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -48,10 +48,26 @@
#define PCIE_RC_CFG_PRIV1_LINK_CAPABILITY 0x04dc
#define PCIE_RC_CFG_PRIV1_LINK_CAPABILITY_ASPM_SUPPORT_MASK 0xc00
+#define PCIE_RC_TL_VDM_CTL0 0x0a20
+#define PCIE_RC_TL_VDM_CTL0_VDM_ENABLED_MASK 0x10000
+#define PCIE_RC_TL_VDM_CTL0_VDM_IGNORETAG_MASK 0x20000
+#define PCIE_RC_TL_VDM_CTL0_VDM_IGNOREVNDRID_MASK 0x40000
+
+#define PCIE_RC_TL_VDM_CTL1 0x0a0c
+#define PCIE_RC_TL_VDM_CTL1_VDM_VNDRID0_MASK 0x0000ffff
+#define PCIE_RC_TL_VDM_CTL1_VDM_VNDRID1_MASK 0xffff0000
+
+#define PCIE_RC_CFG_PRIV1_ROOT_CAP 0x4f8
+#define PCIE_RC_CFG_PRIV1_ROOT_CAP_L1SS_MODE_MASK 0xf8
+
#define PCIE_RC_DL_MDIO_ADDR 0x1100
#define PCIE_RC_DL_MDIO_WR_DATA 0x1104
#define PCIE_RC_DL_MDIO_RD_DATA 0x1108
+#define PCIE_RC_PL_PHY_CTL_15 0x184c
+#define PCIE_RC_PL_PHY_CTL_15_DIS_PLL_PD_MASK 0x400000
+#define PCIE_RC_PL_PHY_CTL_15_PM_CLK_PERIOD_MASK 0xff
+
#define PCIE_MISC_MISC_CTRL 0x4008
#define PCIE_MISC_MISC_CTRL_PCIE_RCB_64B_MODE_MASK 0x80
#define PCIE_MISC_MISC_CTRL_PCIE_RCB_MPS_MODE_MASK 0x400
@@ -82,9 +98,15 @@
#define PCIE_BRCM_MAX_INBOUND_WINS 16
#define PCIE_MISC_RC_BAR1_CONFIG_LO 0x402c
#define PCIE_MISC_RC_BAR1_CONFIG_LO_SIZE_MASK 0x1f
+#define PCIE_MISC_RC_BAR1_CONFIG_HI 0x4030
-#define PCIE_MISC_RC_BAR4_CONFIG_LO 0x40d4
-
+#define PCIE_MISC_RC_BAR2_CONFIG_LO 0x4034
+#define PCIE_MISC_RC_BAR2_CONFIG_LO_SIZE_MASK 0x1f
+#define PCIE_MISC_RC_BAR2_CONFIG_HI 0x4038
+
+#define PCIE_MISC_RC_BAR3_CONFIG_LO 0x403c
+#define PCIE_MISC_RC_BAR3_CONFIG_LO_SIZE_MASK 0x1f
+#define PCIE_MISC_RC_BAR3_CONFIG_HI 0x4040
#define PCIE_MISC_MSI_BAR_CONFIG_LO 0x4044
#define PCIE_MISC_MSI_BAR_CONFIG_HI 0x4048
@@ -93,12 +115,15 @@
#define PCIE_MISC_MSI_DATA_CONFIG_VAL_32 0xffe06540
#define PCIE_MISC_MSI_DATA_CONFIG_VAL_8 0xfff86540
+#define PCIE_MISC_RC_CONFIG_RETRY_TIMEOUT 0x405c
+
#define PCIE_MISC_PCIE_CTRL 0x4064
#define PCIE_MISC_PCIE_CTRL_PCIE_L23_REQUEST_MASK 0x1
#define PCIE_MISC_PCIE_CTRL_PCIE_PERSTB_MASK 0x4
#define PCIE_MISC_PCIE_STATUS 0x4068
#define PCIE_MISC_PCIE_STATUS_PCIE_PORT_MASK 0x80
+#define PCIE_MISC_PCIE_STATUS_PCIE_PORT_MASK_2712 0x40
#define PCIE_MISC_PCIE_STATUS_PCIE_DL_ACTIVE_MASK 0x20
#define PCIE_MISC_PCIE_STATUS_PCIE_PHYLINKUP_MASK 0x10
#define PCIE_MISC_PCIE_STATUS_PCIE_LINK_IN_L23_MASK 0x40
@@ -124,14 +149,82 @@
PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LIMIT_HI + ((win) * 8)
#define PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK 0x2
+#define PCIE_MISC_HARD_PCIE_HARD_DEBUG_PERST_ASSERT_MASK 0x8
+#define PCIE_MISC_HARD_PCIE_HARD_DEBUG_L1SS_ENABLE_MASK 0x200000
#define PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK 0x08000000
#define PCIE_BMIPS_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK 0x00800000
+#define PCIE_CLKREQ_MASK \
+ (PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK | \
+ PCIE_MISC_HARD_PCIE_HARD_DEBUG_L1SS_ENABLE_MASK)
#define PCIE_MISC_UBUS_BAR1_CONFIG_REMAP 0x40ac
#define PCIE_MISC_UBUS_BAR1_CONFIG_REMAP_ACCESS_EN_MASK BIT(0)
#define PCIE_MISC_UBUS_BAR4_CONFIG_REMAP 0x410c
+#define PCIE_MISC_CTRL_1 0x40A0
+#define PCIE_MISC_CTRL_1_OUTBOUND_TC_MASK 0xf
+#define PCIE_MISC_CTRL_1_OUTBOUND_NO_SNOOP_MASK BIT(3)
+#define PCIE_MISC_CTRL_1_OUTBOUND_RO_MASK BIT(4)
+#define PCIE_MISC_CTRL_1_EN_VDM_QOS_CONTROL_MASK BIT(5)
+
+#define PCIE_MISC_UBUS_CTRL 0x40a4
+#define PCIE_MISC_UBUS_CTRL_UBUS_PCIE_REPLY_ERR_DIS_MASK BIT(13)
+#define PCIE_MISC_UBUS_CTRL_UBUS_PCIE_REPLY_DECERR_DIS_MASK BIT(19)
+
+#define PCIE_MISC_UBUS_TIMEOUT 0x40A8
+
+#define PCIE_MISC_UBUS_BAR1_CONFIG_REMAP 0x40ac
+#define PCIE_MISC_UBUS_BAR1_CONFIG_REMAP_ACCESS_ENABLE_MASK BIT(0)
+#define PCIE_MISC_UBUS_BAR1_CONFIG_REMAP_HI 0x40b0
+
+#define PCIE_MISC_UBUS_BAR2_CONFIG_REMAP 0x40b4
+#define PCIE_MISC_UBUS_BAR2_CONFIG_REMAP_ACCESS_ENABLE_MASK BIT(0)
+
+/* Additional RC BARs */
+#define PCIE_MISC_RC_BAR_CONFIG_LO_SIZE_MASK 0x1f
+#define PCIE_MISC_RC_BAR4_CONFIG_LO 0x40d4
+#define PCIE_MISC_RC_BAR4_CONFIG_HI 0x40d8
+/* ... */
+#define PCIE_MISC_RC_BAR10_CONFIG_LO 0x4104
+#define PCIE_MISC_RC_BAR10_CONFIG_HI 0x4108
+
+#define PCIE_MISC_UBUS_BAR_CONFIG_REMAP_ENABLE 0x1
+#define PCIE_MISC_UBUS_BAR_CONFIG_REMAP_LO_MASK 0xfffff000
+#define PCIE_MISC_UBUS_BAR_CONFIG_REMAP_HI_MASK 0xff
+#define PCIE_MISC_UBUS_BAR4_CONFIG_REMAP_LO 0x410c
+#define PCIE_MISC_UBUS_BAR4_CONFIG_REMAP_HI 0x4110
+/* ... */
+#define PCIE_MISC_UBUS_BAR10_CONFIG_REMAP_LO 0x413c
+#define PCIE_MISC_UBUS_BAR10_CONFIG_REMAP_HI 0x4140
+
+/* AXI priority forwarding - automatic level-based */
+#define PCIE_MISC_TC_QUEUE_TO_QOS_MAP(x) (0x4160 - (x) * 4)
+/* Defined in quarter-fullness */
+#define QUEUE_THRESHOLD_34_TO_QOS_MAP_SHIFT 12
+#define QUEUE_THRESHOLD_23_TO_QOS_MAP_SHIFT 8
+#define QUEUE_THRESHOLD_12_TO_QOS_MAP_SHIFT 4
+#define QUEUE_THRESHOLD_01_TO_QOS_MAP_SHIFT 0
+#define QUEUE_THRESHOLD_MASK 0xf
+
+/* VDM messages indexing TCs to AXI priorities */
+/* Indexes 8-15 */
+#define PCIE_MISC_VDM_PRIORITY_TO_QOS_MAP_HI 0x4164
+/* Indexes 0-7 */
+#define PCIE_MISC_VDM_PRIORITY_TO_QOS_MAP_LO 0x4168
+#define VDM_PRIORITY_TO_QOS_MAP_SHIFT(x) (4 * (x))
+#define VDM_PRIORITY_TO_QOS_MAP_MASK 0xf
+
+#define PCIE_MISC_AXI_INTF_CTRL 0x416C
+#define AXI_EN_RCLK_QOS_ARRAY_FIX BIT(13)
+#define AXI_EN_QOS_UPDATE_TIMING_FIX BIT(12)
+#define AXI_DIS_QOS_GATING_IN_MASTER BIT(11)
+#define AXI_REQFIFO_EN_QOS_PROPAGATION BIT(7)
+#define AXI_BRIDGE_LOW_LATENCY_MODE BIT(6)
+#define AXI_MASTER_MAX_OUTSTANDING_REQUESTS_MASK 0x3f
+
+#define PCIE_MISC_AXI_READ_ERROR_DATA 0x4170
+
#define PCIE_MSI_INTR2_BASE 0x4500
/* Offsets from INTR2_CPU and MSI_INTR2 BASE offsets */
@@ -220,6 +313,7 @@ enum pcie_soc_base {
BCM7425,
BCM7435,
BCM7712,
+ BCM2712,
};
struct inbound_win {
@@ -251,7 +345,7 @@ struct brcm_msi {
struct mutex lock; /* guards the alloc/free operations */
u64 target_addr;
int irq;
- DECLARE_BITMAP(used, BRCM_INT_PCI_MSI_NR);
+ DECLARE_BITMAP(used, 64);
bool legacy;
/* Some chips have MSIs in bits [31..24] of a shared register. */
int legacy_shift;
@@ -285,6 +379,10 @@ struct brcm_pcie {
bool ep_wakeup_capable;
bool has_phy;
u8 num_inbound_wins;
+ bool l1ss;
+ bool rcb_mps_mode;
+ u32 qos_map;
+ u32 tperst_clk_ms;
};
static inline bool is_bmips(const struct brcm_pcie *pcie)
@@ -303,8 +401,8 @@ static int brcm_pcie_encode_ibar_size(u6
if (log2_in >= 12 && log2_in <= 15)
/* Covers 4KB to 32KB (inclusive) */
return (log2_in - 12) + 0x1c;
- else if (log2_in >= 16 && log2_in <= 35)
- /* Covers 64KB to 32GB, (inclusive) */
+ else if (log2_in >= 16 && log2_in <= 36)
+ /* Covers 64KB to 64GB, (inclusive) */
return log2_in - 15;
/* Something is awry so disable */
return 0;
@@ -393,12 +491,43 @@ static int brcm_pcie_set_ssc(struct brcm
return ssc && pll ? 0 : -EIO;
}
+static void brcm_pcie_munge_pll(struct brcm_pcie *pcie)
+{
+ //print "MDIO block 0x1600 written per Dannys instruction"
+ //tmp = pcie_mdio_write(phyad, &h16&, &h50b9&)
+ //tmp = pcie_mdio_write(phyad, &h17&, &hbd1a&)
+ //tmp = pcie_mdio_write(phyad, &h1b&, &h5030&)
+ //tmp = pcie_mdio_write(phyad, &h1e&, &h0007&)
+
+ u32 tmp;
+ int ret, i;
+ u8 regs[] = { 0x16, 0x17, 0x18, 0x19, 0x1b, 0x1c, 0x1e };
+ u16 data[] = { 0x50b9, 0xbda1, 0x0094, 0x97b4, 0x5030, 0x5030, 0x0007 };
+
+ ret = brcm_pcie_mdio_write(pcie->base, MDIO_PORT0, SET_ADDR_OFFSET,
+ 0x1600);
+ for (i = 0; i < ARRAY_SIZE(regs); i++) {
+ brcm_pcie_mdio_read(pcie->base, MDIO_PORT0, regs[i], &tmp);
+ dev_dbg(pcie->dev, "PCIE MDIO pre_refclk 0x%02x = 0x%04x\n",
+ regs[i], tmp);
+ }
+ for (i = 0; i < ARRAY_SIZE(regs); i++) {
+ brcm_pcie_mdio_write(pcie->base, MDIO_PORT0, regs[i], data[i]);
+ brcm_pcie_mdio_read(pcie->base, MDIO_PORT0, regs[i], &tmp);
+ dev_dbg(pcie->dev, "PCIE MDIO post_refclk 0x%02x = 0x%04x\n",
+ regs[i], tmp);
+ }
+ usleep_range(100, 200);
+}
+
/* Limits operation to a specific generation (1, 2, or 3) */
static void brcm_pcie_set_gen(struct brcm_pcie *pcie, int gen)
{
u16 lnkctl2 = readw(pcie->base + BRCM_PCIE_CAP_REGS + PCI_EXP_LNKCTL2);
u32 lnkcap = readl(pcie->base + BRCM_PCIE_CAP_REGS + PCI_EXP_LNKCAP);
+ dev_info(pcie->dev, "Forcing gen %d\n", pcie->gen);
+
lnkcap = (lnkcap & ~PCI_EXP_LNKCAP_SLS) | gen;
writel(lnkcap, pcie->base + BRCM_PCIE_CAP_REGS + PCI_EXP_LNKCAP);
@@ -450,6 +579,73 @@ static void brcm_pcie_set_outbound_win(s
writel(tmp, pcie->base + PCIE_MEM_WIN0_LIMIT_HI(win));
}
+static void brcm_pcie_set_tc_qos(struct brcm_pcie *pcie)
+{
+ int i;
+ u32 reg;
+
+ if (pcie->soc_base != BCM2712)
+ return;
+
+ /* Disable broken QOS forwarding search. Set chicken bits for 2712D0 */
+ reg = readl(pcie->base + PCIE_MISC_AXI_INTF_CTRL);
+ reg &= ~AXI_REQFIFO_EN_QOS_PROPAGATION;
+ reg |= AXI_EN_RCLK_QOS_ARRAY_FIX | AXI_EN_QOS_UPDATE_TIMING_FIX |
+ AXI_DIS_QOS_GATING_IN_MASTER;
+ writel(reg, pcie->base + PCIE_MISC_AXI_INTF_CTRL);
+
+ /*
+ * If the QOS_UPDATE_TIMING_FIX bit is Reserved-0, then this is a
+ * 2712C1 chip, or a single-lane RC. Use the best-effort alternative
+ * which is to partially throttle AXI requests in-flight to the SDC.
+ */
+ reg = readl(pcie->base + PCIE_MISC_AXI_INTF_CTRL);
+ if (!(reg & AXI_EN_QOS_UPDATE_TIMING_FIX)) {
+ reg &= ~AXI_MASTER_MAX_OUTSTANDING_REQUESTS_MASK;
+ reg |= 15;
+ writel(reg, pcie->base + PCIE_MISC_AXI_INTF_CTRL);
+ }
+
+ /* Disable VDM reception by default - QoS map defaults to 0 */
+ reg = readl(pcie->base + PCIE_MISC_CTRL_1);
+ reg &= ~PCIE_MISC_CTRL_1_EN_VDM_QOS_CONTROL_MASK;
+ writel(reg, pcie->base + PCIE_MISC_CTRL_1);
+
+ if (!of_property_read_u32(pcie->np, "brcm,fifo-qos-map", &pcie->qos_map)) {
+ /*
+ * Backpressure mode - bottom 4 nibbles are QoS for each
+ * quartile of FIFO level. Each TC gets the same map, because
+ * this mode is intended for nonrealtime EPs.
+ */
+
+ pcie->qos_map &= 0x0000ffff;
+ for (i = 0; i < 8; i++)
+ writel(pcie->qos_map, pcie->base + PCIE_MISC_TC_QUEUE_TO_QOS_MAP(i));
+
+ return;
+ }
+
+ if (!of_property_read_u32(pcie->np, "brcm,vdm-qos-map", &pcie->qos_map)) {
+
+ reg = readl(pcie->base + PCIE_MISC_CTRL_1);
+ reg |= PCIE_MISC_CTRL_1_EN_VDM_QOS_CONTROL_MASK;
+ writel(reg, pcie->base + PCIE_MISC_CTRL_1);
+
+ /* No forwarding means no point separating panic priorities from normal */
+ writel(pcie->qos_map, pcie->base + PCIE_MISC_VDM_PRIORITY_TO_QOS_MAP_LO);
+ writel(pcie->qos_map, pcie->base + PCIE_MISC_VDM_PRIORITY_TO_QOS_MAP_HI);
+
+ /* Match Vendor ID of 0 */
+ writel(0, pcie->base + PCIE_RC_TL_VDM_CTL1);
+ /* Forward VDMs to priority interface - at least the rx counters work */
+ reg = readl(pcie->base + PCIE_RC_TL_VDM_CTL0);
+ reg |= PCIE_RC_TL_VDM_CTL0_VDM_ENABLED_MASK |
+ PCIE_RC_TL_VDM_CTL0_VDM_IGNORETAG_MASK |
+ PCIE_RC_TL_VDM_CTL0_VDM_IGNOREVNDRID_MASK;
+ writel(reg, pcie->base + PCIE_RC_TL_VDM_CTL0);
+ }
+}
+
static struct irq_chip brcm_msi_irq_chip = {
.name = "BRCM STB PCIe MSI",
.irq_ack = irq_chip_ack_parent,
@@ -458,8 +654,8 @@ static struct irq_chip brcm_msi_irq_chip
};
static struct msi_domain_info brcm_msi_domain_info = {
- .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
- MSI_FLAG_NO_AFFINITY | MSI_FLAG_MULTI_PCI_MSI,
+ .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
+ MSI_FLAG_NO_AFFINITY | MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX),
.chip = &brcm_msi_irq_chip,
};
@@ -478,10 +674,23 @@ static void brcm_pcie_msi_isr(struct irq
status = readl(msi->intr_base + MSI_INT_STATUS);
status >>= msi->legacy_shift;
- for_each_set_bit(bit, &status, msi->nr) {
- int ret;
- ret = generic_handle_domain_irq(msi->inner_domain, bit);
- if (ret)
+ for_each_set_bit(bit, &status, BRCM_INT_PCI_MSI_NR/*msi->nr*/) {
+ unsigned long virq;
+ bool found = false;
+
+ virq = irq_find_mapping(msi->inner_domain, bit);
+ if (virq) {
+ found = true;
+ dev_dbg(dev, "MSI -> %ld\n", virq);
+ generic_handle_irq(virq);
+ }
+ virq = irq_find_mapping(msi->inner_domain, bit + 32);
+ if (virq) {
+ found = true;
+ dev_dbg(dev, "MSI -> %ld\n", virq);
+ generic_handle_irq(virq);
+ }
+ if (!found)
dev_dbg(dev, "unexpected MSI\n");
}
@@ -494,13 +703,13 @@ static void brcm_msi_compose_msi_msg(str
msg->address_lo = lower_32_bits(msi->target_addr);
msg->address_hi = upper_32_bits(msi->target_addr);
- msg->data = (0xffff & PCIE_MISC_MSI_DATA_CONFIG_VAL_32) | data->hwirq;
+ msg->data = (0xffff & PCIE_MISC_MSI_DATA_CONFIG_VAL_32) | (data->hwirq & 0x1f);
}
static void brcm_msi_ack_irq(struct irq_data *data)
{
struct brcm_msi *msi = irq_data_get_irq_chip_data(data);
- const int shift_amt = data->hwirq + msi->legacy_shift;
+ const int shift_amt = (data->hwirq & 0x1f) + msi->legacy_shift;
writel(1 << shift_amt, msi->intr_base + MSI_INT_CLR);
}
@@ -660,7 +869,7 @@ static int brcm_pcie_enable_msi(struct b
msi->legacy_shift = 24;
} else {
msi->intr_base = msi->base + PCIE_MSI_INTR2_BASE;
- msi->nr = BRCM_INT_PCI_MSI_NR;
+ msi->nr = 64; //BRCM_INT_PCI_MSI_NR;
msi->legacy_shift = 0;
}
@@ -682,6 +891,9 @@ static bool brcm_pcie_rc_mode(struct brc
void __iomem *base = pcie->base;
u32 val = readl(base + PCIE_MISC_PCIE_STATUS);
+ if (pcie->soc_base == BCM2712)
+ return !!FIELD_GET(PCIE_MISC_PCIE_STATUS_PCIE_PORT_MASK_2712, val) | 1; //XXX
+
return !!FIELD_GET(PCIE_MISC_PCIE_STATUS_PCIE_PORT_MASK, val);
}
@@ -774,6 +986,21 @@ static int brcm_pcie_bridge_sw_init_set_
return 0;
}
+static int brcm_pcie_bridge_sw_init_set_2712(struct brcm_pcie *pcie, u32 val)
+{
+ int ret;
+
+ if (WARN_ONCE(!pcie->bridge_reset, "missing bridge reset controller\n"))
+ return -EINVAL;
+
+ if (val)
+ ret = reset_control_assert(pcie->bridge_reset);
+ else
+ ret = reset_control_deassert(pcie->bridge_reset);
+
+ return ret;
+}
+
static int brcm_pcie_perst_set_4908(struct brcm_pcie *pcie, u32 val)
{
int ret;
@@ -804,6 +1031,18 @@ static int brcm_pcie_perst_set_7278(stru
return 0;
}
+static int brcm_pcie_perst_set_2712(struct brcm_pcie *pcie, u32 val)
+{
+ u32 tmp;
+
+ /* Perst bit has moved and assert value is 0 */
+ tmp = readl(pcie->base + PCIE_MISC_PCIE_CTRL);
+ u32p_replace_bits(&tmp, !val, PCIE_MISC_PCIE_CTRL_PCIE_PERSTB_MASK);
+ writel(tmp, pcie->base + PCIE_MISC_PCIE_CTRL);
+
+ return 0;
+}
+
static int brcm_pcie_perst_set_generic(struct brcm_pcie *pcie, u32 val)
{
u32 tmp;
@@ -815,6 +1054,8 @@ static int brcm_pcie_perst_set_generic(s
return 0;
}
+
+#if 0
static void add_inbound_win(struct inbound_win *b, u8 *count, u64 size,
u64 cpu_addr, u64 pci_offset)
{
@@ -1017,17 +1258,137 @@ static void set_inbound_win_registers(st
}
}
}
+#endif
+
+static int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
+ u64 *rc_bar2_size,
+ u64 *rc_bar2_offset)
+{
+ struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
+ struct resource_entry *entry;
+ struct device *dev = pcie->dev;
+ u64 lowest_pcie_addr = ~(u64)0;
+ int ret, i = 0;
+ u64 size = 0;
+
+ resource_list_for_each_entry(entry, &bridge->dma_ranges) {
+ u64 pcie_beg = entry->res->start - entry->offset;
+
+ size += entry->res->end - entry->res->start + 1;
+ if (pcie_beg < lowest_pcie_addr)
+ lowest_pcie_addr = pcie_beg;
+ if (pcie->soc_base == BCM2711 || pcie->soc_base == BCM2712)
+ break; // Only consider the first entry
+ }
+
+ if (lowest_pcie_addr == ~(u64)0) {
+ dev_err(dev, "DT node has no dma-ranges\n");
+ return -EINVAL;
+ }
+
+ ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1,
+ PCIE_BRCM_MAX_MEMC);
+
+ if (ret <= 0) {
+ /* Make an educated guess */
+ pcie->num_memc = 1;
+ pcie->memc_size[0] = 1ULL << fls64(size - 1);
+ } else {
+ pcie->num_memc = ret;
+ }
+
+ /* Each memc is viewed through a "port" that is a power of 2 */
+ for (i = 0, size = 0; i < pcie->num_memc; i++)
+ size += pcie->memc_size[i];
+
+ /* System memory starts at this address in PCIe-space */
+ *rc_bar2_offset = lowest_pcie_addr;
+ /* The sum of all memc views must also be a power of 2 */
+ *rc_bar2_size = 1ULL << fls64(size - 1);
+
+ /*
+ * We validate the inbound memory view even though we should trust
+ * whatever the device-tree provides. This is because of an HW issue on
+ * early Raspberry Pi 4's revisions (bcm2711). It turns out its
+ * firmware has to dynamically edit dma-ranges due to a bug on the
+ * PCIe controller integration, which prohibits any access above the
+ * lower 3GB of memory. Given this, we decided to keep the dma-ranges
+ * in check, avoiding hard to debug device-tree related issues in the
+ * future:
+ *
+ * The PCIe host controller by design must set the inbound viewport to
+ * be a contiguous arrangement of all of the system's memory. In
+ * addition, its size mut be a power of two. To further complicate
+ * matters, the viewport must start on a pcie-address that is aligned
+ * on a multiple of its size. If a portion of the viewport does not
+ * represent system memory -- e.g. 3GB of memory requires a 4GB
+ * viewport -- we can map the outbound memory in or after 3GB and even
+ * though the viewport will overlap the outbound memory the controller
+ * will know to send outbound memory downstream and everything else
+ * upstream.
+ *
+ * For example:
+ *
+ * - The best-case scenario, memory up to 3GB, is to place the inbound
+ * region in the first 4GB of pcie-space, as some legacy devices can
+ * only address 32bits. We would also like to put the MSI under 4GB
+ * as well, since some devices require a 32bit MSI target address.
+ *
+ * - If the system memory is 4GB or larger we cannot start the inbound
+ * region at location 0 (since we have to allow some space for
+ * outbound memory @ 3GB). So instead it will start at the 1x
+ * multiple of its size
+ */
+ if (!*rc_bar2_size || (*rc_bar2_offset & (*rc_bar2_size - 1)) ||
+ (*rc_bar2_offset < SZ_4G && *rc_bar2_offset > SZ_2G)) {
+ dev_err(dev, "Invalid rc_bar2_offset/size: size 0x%llx, off 0x%llx\n",
+ *rc_bar2_size, *rc_bar2_offset);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int brcm_pcie_get_rc_bar_n(struct brcm_pcie *pcie,
+ int idx,
+ u64 *rc_bar_cpu,
+ u64 *rc_bar_size,
+ u64 *rc_bar_pci)
+{
+ struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
+ struct resource_entry *entry;
+ int i = 0;
+
+ resource_list_for_each_entry(entry, &bridge->dma_ranges) {
+ if (i == idx) {
+ *rc_bar_cpu = entry->res->start;
+ *rc_bar_size = entry->res->end - entry->res->start + 1;
+ *rc_bar_pci = entry->res->start - entry->offset;
+ return 0;
+ }
+
+ i++;
+ }
+
+ return -EINVAL;
+}
static int brcm_pcie_setup(struct brcm_pcie *pcie)
{
+#if 0
struct inbound_win inbound_wins[PCIE_BRCM_MAX_INBOUND_WINS];
+#endif
+ u64 rc_bar2_offset, rc_bar2_size;
void __iomem *base = pcie->base;
struct pci_host_bridge *bridge;
struct resource_entry *entry;
u32 tmp, burst, aspm_support;
u8 num_out_wins = 0;
+#if 0
int num_inbound_wins = 0;
+#endif
int memc, ret;
+ int count, i;
/* Reset the bridge */
ret = pcie->bridge_sw_init_set(pcie, 1);
@@ -1059,6 +1420,17 @@ static int brcm_pcie_setup(struct brcm_p
/* Wait for SerDes to be stable */
usleep_range(100, 200);
+ if (pcie->soc_base == BCM2712) {
+ /* Allow a 54MHz (xosc) refclk source */
+ brcm_pcie_munge_pll(pcie);
+ /* Fix for L1SS errata */
+ tmp = readl(base + PCIE_RC_PL_PHY_CTL_15);
+ tmp &= ~PCIE_RC_PL_PHY_CTL_15_PM_CLK_PERIOD_MASK;
+ /* PM clock period is 18.52ns (round down) */
+ tmp |= 0x12;
+ writel(tmp, base + PCIE_RC_PL_PHY_CTL_15);
+ }
+
/*
* SCB_MAX_BURST_SIZE is a two bit field. For GENERIC chips it
* is encoded as 0=128, 1=256, 2=512, 3=Rsvd, for BCM7278 it
@@ -1068,6 +1440,8 @@ static int brcm_pcie_setup(struct brcm_p
burst = 0x1; /* 256 bytes */
else if (pcie->soc_base == BCM2711)
burst = 0x0; /* 128 bytes */
+ else if (pcie->soc_base == BCM2712)
+ burst = 0x1; /* 128 bytes */
else if (pcie->soc_base == BCM7278)
burst = 0x3; /* 512 bytes */
else
@@ -1075,27 +1449,38 @@ static int brcm_pcie_setup(struct brcm_p
/*
* Set SCB_MAX_BURST_SIZE, CFG_READ_UR_MODE, SCB_ACCESS_EN,
- * RCB_MPS_MODE, RCB_64B_MODE
+ * RCB_MPS_MODE
*/
tmp = readl(base + PCIE_MISC_MISC_CTRL);
u32p_replace_bits(&tmp, 1, PCIE_MISC_MISC_CTRL_SCB_ACCESS_EN_MASK);
u32p_replace_bits(&tmp, 1, PCIE_MISC_MISC_CTRL_CFG_READ_UR_MODE_MASK);
u32p_replace_bits(&tmp, burst, PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK);
- u32p_replace_bits(&tmp, 1, PCIE_MISC_MISC_CTRL_PCIE_RCB_MPS_MODE_MASK);
- u32p_replace_bits(&tmp, 1, PCIE_MISC_MISC_CTRL_PCIE_RCB_64B_MODE_MASK);
+ if (pcie->rcb_mps_mode)
+ u32p_replace_bits(&tmp, 1, PCIE_MISC_MISC_CTRL_PCIE_RCB_MPS_MODE_MASK);
writel(tmp, base + PCIE_MISC_MISC_CTRL);
- num_inbound_wins = brcm_pcie_get_inbound_wins(pcie, inbound_wins);
- if (num_inbound_wins < 0)
- return num_inbound_wins;
+ brcm_pcie_set_tc_qos(pcie);
- set_inbound_win_registers(pcie, inbound_wins, num_inbound_wins);
+ ret = brcm_pcie_get_rc_bar2_size_and_offset(pcie, &rc_bar2_size,
+ &rc_bar2_offset);
+ if (ret)
+ return ret;
+
+ tmp = lower_32_bits(rc_bar2_offset);
+ u32p_replace_bits(&tmp, brcm_pcie_encode_ibar_size(rc_bar2_size),
+ PCIE_MISC_RC_BAR2_CONFIG_LO_SIZE_MASK);
+ writel(tmp, base + PCIE_MISC_RC_BAR2_CONFIG_LO);
+ writel(upper_32_bits(rc_bar2_offset),
+ base + PCIE_MISC_RC_BAR2_CONFIG_HI);
if (!brcm_pcie_rc_mode(pcie)) {
dev_err(pcie->dev, "PCIe RC controller misconfigured as Endpoint\n");
return -EINVAL;
}
+ tmp = readl(base + PCIE_MISC_UBUS_BAR2_CONFIG_REMAP);
+ u32p_replace_bits(&tmp, 1, PCIE_MISC_UBUS_BAR2_CONFIG_REMAP_ACCESS_ENABLE_MASK);
+ writel(tmp, base + PCIE_MISC_UBUS_BAR2_CONFIG_REMAP);
tmp = readl(base + PCIE_MISC_MISC_CTRL);
for (memc = 0; memc < pcie->num_memc; memc++) {
u32 scb_size_val = ilog2(pcie->memc_size[memc]) - 15;
@@ -1109,6 +1494,29 @@ static int brcm_pcie_setup(struct brcm_p
}
writel(tmp, base + PCIE_MISC_MISC_CTRL);
+ if (pcie->soc_base == BCM2712) {
+ /* Suppress AXI error responses and return 1s for read failures */
+ tmp = readl(base + PCIE_MISC_UBUS_CTRL);
+ u32p_replace_bits(&tmp, 1, PCIE_MISC_UBUS_CTRL_UBUS_PCIE_REPLY_ERR_DIS_MASK);
+ u32p_replace_bits(&tmp, 1, PCIE_MISC_UBUS_CTRL_UBUS_PCIE_REPLY_DECERR_DIS_MASK);
+ writel(tmp, base + PCIE_MISC_UBUS_CTRL);
+ writel(0xffffffff, base + PCIE_MISC_AXI_READ_ERROR_DATA);
+
+ /*
+ * Adjust timeouts. The UBUS timeout also affects CRS
+ * completion retries, as the request will get terminated if
+ * either timeout expires, so both have to be a large value
+ * (in clocks of 750MHz).
+ * Set UBUS timeout to 250ms, then set RC config retry timeout
+ * to be ~240ms.
+ *
+ * Setting CRSVis=1 will stop the core from blocking on a CRS
+ * response, but does require the device to be well-behaved...
+ */
+ writel(0xB2D0000, base + PCIE_MISC_UBUS_TIMEOUT);
+ writel(0xABA0000, base + PCIE_MISC_RC_CONFIG_RETRY_TIMEOUT);
+ }
+
/*
* We ideally want the MSI target address to be located in the 32bit
* addressable memory area. Some devices might depend on it. This is
@@ -1116,12 +1524,20 @@ static int brcm_pcie_setup(struct brcm_p
* 4GB or when the inbound area is smaller than 4GB (taking into
* account the rounding-up we're forced to perform).
*/
- if (inbound_wins[2].pci_offset >= SZ_4G ||
- (inbound_wins[2].size + inbound_wins[2].pci_offset) < SZ_4G)
+ if (rc_bar2_offset >= SZ_4G || (rc_bar2_size + rc_bar2_offset) < SZ_4G)
pcie->msi_target_addr = BRCM_MSI_TARGET_ADDR_LT_4GB;
else
pcie->msi_target_addr = BRCM_MSI_TARGET_ADDR_GT_4GB;
+ /* disable the PCIe->GISB memory window (RC_BAR1) */
+ tmp = readl(base + PCIE_MISC_RC_BAR1_CONFIG_LO);
+ tmp &= ~PCIE_MISC_RC_BAR1_CONFIG_LO_SIZE_MASK;
+ writel(tmp, base + PCIE_MISC_RC_BAR1_CONFIG_LO);
+
+ /* disable the PCIe->SCB memory window (RC_BAR3) */
+ tmp = readl(base + PCIE_MISC_RC_BAR3_CONFIG_LO);
+ tmp &= ~PCIE_MISC_RC_BAR3_CONFIG_LO_SIZE_MASK;
+ writel(tmp, base + PCIE_MISC_RC_BAR3_CONFIG_LO);
/* Don't advertise L0s capability if 'aspm-no-l0s' */
aspm_support = PCIE_LINK_STATE_L1;
@@ -1132,6 +1548,33 @@ static int brcm_pcie_setup(struct brcm_p
PCIE_RC_CFG_PRIV1_LINK_CAPABILITY_ASPM_SUPPORT_MASK);
writel(tmp, base + PCIE_RC_CFG_PRIV1_LINK_CAPABILITY);
+ /* program additional inbound windows (RC_BAR4..RC_BAR10) */
+ count = (pcie->soc_base == BCM2712) ? 7 : 0;
+ for (i = 0; i < count; i++) {
+ u64 bar_cpu, bar_size, bar_pci;
+
+ ret = brcm_pcie_get_rc_bar_n(pcie, 1 + i, &bar_cpu, &bar_size,
+ &bar_pci);
+ if (ret)
+ break;
+
+ tmp = lower_32_bits(bar_pci);
+ u32p_replace_bits(&tmp, brcm_pcie_encode_ibar_size(bar_size),
+ PCIE_MISC_RC_BAR_CONFIG_LO_SIZE_MASK);
+ writel(tmp, base + PCIE_MISC_RC_BAR4_CONFIG_LO + i * 8);
+ writel(upper_32_bits(bar_pci),
+ base + PCIE_MISC_RC_BAR4_CONFIG_HI + i * 8);
+
+ tmp = upper_32_bits(bar_cpu) &
+ PCIE_MISC_UBUS_BAR_CONFIG_REMAP_HI_MASK;
+ writel(tmp,
+ base + PCIE_MISC_UBUS_BAR4_CONFIG_REMAP_HI + i * 8);
+ tmp = lower_32_bits(bar_cpu) &
+ PCIE_MISC_UBUS_BAR_CONFIG_REMAP_LO_MASK;
+ writel(tmp | PCIE_MISC_UBUS_BAR_CONFIG_REMAP_ENABLE,
+ base + PCIE_MISC_UBUS_BAR4_CONFIG_REMAP_LO + i * 8);
+ }
+
/*
* For config space accesses on the RC, show the right class for
* a PCIe-PCIe bridge (the default setting is to be EP mode).
@@ -1181,6 +1624,87 @@ static int brcm_pcie_setup(struct brcm_p
return 0;
}
+/*
+ * This extends the timeout period for an access to an internal bus. This
+ * access timeout may occur during L1SS sleep periods, even without the
+ * presence of a PCIe access.
+ */
+static void brcm_extend_rbus_timeout(struct brcm_pcie *pcie)
+{
+ /* TIMEOUT register is two registers before RGR1_SW_INIT_1 */
+ const unsigned int REG_OFFSET = PCIE_RGR1_SW_INIT_1(pcie) - 8;
+ u32 timeout_us = 4000000; /* 4 seconds, our setting for L1SS */
+
+ /* 7712 does not have this (RGR1) timer */
+ if (pcie->soc_base == BCM7712 || pcie->soc_base == BCM2712)
+ return;
+
+ /* Each unit in timeout register is 1/216,000,000 seconds */
+ writel(216 * timeout_us, pcie->base + REG_OFFSET);
+}
+
+static void brcm_config_clkreq(struct brcm_pcie *pcie)
+{
+ static const char err_msg[] = "invalid 'brcm,clkreq-mode' DT string\n";
+ const char *mode = "default";
+ u32 clkreq_cntl;
+ int ret, tmp;
+
+ ret = of_property_read_string(pcie->np, "brcm,clkreq-mode", &mode);
+ if (ret && ret != -EINVAL) {
+ dev_err(pcie->dev, err_msg);
+ mode = "safe";
+ }
+
+ /* Start out assuming safe mode (both mode bits cleared) */
+ clkreq_cntl = readl(pcie->base + HARD_DEBUG(pcie));
+ clkreq_cntl &= ~PCIE_CLKREQ_MASK;
+
+ if (strcmp(mode, "no-l1ss") == 0) {
+ /*
+ * "no-l1ss" -- Provides Clock Power Management, L0s, and
+ * L1, but cannot provide L1 substate (L1SS) power
+ * savings. If the downstream device connected to the RC is
+ * L1SS capable AND the OS enables L1SS, all PCIe traffic
+ * may abruptly halt, potentially hanging the system.
+ */
+ clkreq_cntl |= PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK;
+ /*
+ * We want to un-advertise L1 substates because if the OS
+ * tries to configure the controller into using L1 substate
+ * power savings it may fail or hang when the RC HW is in
+ * "no-l1ss" mode.
+ */
+ tmp = readl(pcie->base + PCIE_RC_CFG_PRIV1_ROOT_CAP);
+ u32p_replace_bits(&tmp, 2, PCIE_RC_CFG_PRIV1_ROOT_CAP_L1SS_MODE_MASK);
+ writel(tmp, pcie->base + PCIE_RC_CFG_PRIV1_ROOT_CAP);
+
+ } else if (strcmp(mode, "default") == 0) {
+ /*
+ * "default" -- Provides L0s, L1, and L1SS, but not
+ * compliant to provide Clock Power Management;
+ * specifically, may not be able to meet the Tclron max
+ * timing of 400ns as specified in "Dynamic Clock Control",
+ * section 3.2.5.2.2 of the PCIe spec. This situation is
+ * atypical and should happen only with older devices.
+ */
+ clkreq_cntl |= PCIE_MISC_HARD_PCIE_HARD_DEBUG_L1SS_ENABLE_MASK;
+ brcm_extend_rbus_timeout(pcie);
+
+ } else {
+ /*
+ * "safe" -- No power savings; refclk is driven by RC
+ * unconditionally.
+ */
+ if (strcmp(mode, "safe") != 0)
+ dev_err(pcie->dev, err_msg);
+ mode = "safe";
+ }
+ writel(clkreq_cntl, pcie->base + HARD_DEBUG(pcie));
+
+ dev_info(pcie->dev, "clkreq-mode set to %s\n", mode);
+}
+
static int brcm_pcie_start_link(struct brcm_pcie *pcie)
{
struct device *dev = pcie->dev;
@@ -1188,12 +1712,31 @@ static int brcm_pcie_start_link(struct b
u16 nlw, cls, lnksta;
bool ssc_good = false;
u32 tmp;
+ u16 tmp16;
int ret, i;
/* Unassert the fundamental reset */
- ret = pcie->perst_set(pcie, 0);
- if (ret)
- return ret;
+ if (pcie->tperst_clk_ms) {
+ /*
+ * Increase Tperst_clk time by forcing PERST# output low while
+ * the internal reset is released, so the PLL generates stable
+ * refclk output further in advance of PERST# deassertion.
+ */
+ tmp = readl(base + HARD_DEBUG(pcie));
+ u32p_replace_bits(&tmp, 1, PCIE_MISC_HARD_PCIE_HARD_DEBUG_PERST_ASSERT_MASK);
+ writel(tmp, base + HARD_DEBUG(pcie));
+
+ pcie->perst_set(pcie, 0);
+ msleep(pcie->tperst_clk_ms);
+
+ tmp = readl(base + HARD_DEBUG(pcie));
+ u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_PERST_ASSERT_MASK);
+ writel(tmp, base + HARD_DEBUG(pcie));
+ } else {
+ ret = pcie->perst_set(pcie, 0);
+ if (ret)
+ return ret;
+ }
/*
* Wait for 100ms after PERST# deassertion; see PCIe CEM specification
@@ -1214,6 +1757,8 @@ static int brcm_pcie_start_link(struct b
return -ENODEV;
}
+ brcm_config_clkreq(pcie);
+
if (pcie->gen)
brcm_pcie_set_gen(pcie, pcie->gen);
@@ -1233,13 +1778,15 @@ static int brcm_pcie_start_link(struct b
ssc_good ? "(SSC)" : "(!SSC)");
/*
- * Refclk from RC should be gated with CLKREQ# input when ASPM L0s,L1
- * is enabled => setting the CLKREQ_DEBUG_ENABLE field to 1.
+ * RootCtl bits are reset by perst_n, which undoes pci_enable_crs()
+ * called prior to pci_add_new_bus() during probe. Re-enable here.
*/
- tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
- tmp |= PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK;
- writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
-
+ tmp16 = readw(base + BRCM_PCIE_CAP_REGS + PCI_EXP_RTCAP);
+ if (tmp16 & PCI_EXP_RTCAP_CRSVIS) {
+ tmp16 = readw(base + BRCM_PCIE_CAP_REGS + PCI_EXP_RTCTL);
+ u16p_replace_bits(&tmp16, 1, PCI_EXP_RTCTL_CRSSVE);
+ writew(tmp16, base + BRCM_PCIE_CAP_REGS + PCI_EXP_RTCTL);
+ }
return 0;
}
@@ -1413,6 +1960,12 @@ static int brcm_pcie_turn_off(struct brc
u32p_replace_bits(&tmp, 1, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
writel(tmp, base + HARD_DEBUG(pcie));
+ /*
+ * Shutting down this bridge on pcie1 means accesses to rescal block
+ * will hang the chip if another RC wants to assert/deassert rescal.
+ */
+ if (pcie->soc_base == BCM2712)
+ return 0;
/* Shutdown PCIe bridge */
ret = pcie->bridge_sw_init_set(pcie, 1);
@@ -1608,6 +2161,13 @@ static const int pcie_offsets_bcm7712[]
[PCIE_INTR2_CPU_BASE] = 0x4400,
};
+static const int pcie_offsets_bcm2712[] = {
+ [EXT_CFG_INDEX] = 0x9000,
+ [EXT_CFG_DATA] = 0x9004,
+ [PCIE_HARD_DEBUG] = 0x4304,
+ [PCIE_INTR2_CPU_BASE] = 0x4400,
+};
+
static const struct pcie_cfg_data generic_cfg = {
.offsets = pcie_offsets,
.soc_base = GENERIC,
@@ -1673,6 +2233,13 @@ static const struct pcie_cfg_data bcm771
.num_inbound_wins = 10,
};
+static const struct pcie_cfg_data bcm2712_cfg = {
+ .offsets = pcie_offsets_bcm2712,
+ .perst_set = brcm_pcie_perst_set_2712,
+ .bridge_sw_init_set = brcm_pcie_bridge_sw_init_set_2712,
+ .soc_base = BCM2712,
+};
+
static const struct of_device_id brcm_pcie_match[] = {
{ .compatible = "brcm,bcm2711-pcie", .data = &bcm2711_cfg },
{ .compatible = "brcm,bcm4908-pcie", .data = &bcm4908_cfg },
@@ -1683,6 +2250,7 @@ static const struct of_device_id brcm_pc
{ .compatible = "brcm,bcm7435-pcie", .data = &bcm7435_cfg },
{ .compatible = "brcm,bcm7445-pcie", .data = &generic_cfg },
{ .compatible = "brcm,bcm7712-pcie", .data = &bcm7712_cfg },
+ { .compatible = "brcm,bcm2712-pcie", .data = &bcm2712_cfg },
{},
};
@@ -1742,6 +2310,9 @@ static int brcm_pcie_probe(struct platfo
pcie->gen = (ret < 0) ? 0 : ret;
pcie->ssc = of_property_read_bool(np, "brcm,enable-ssc");
+ pcie->l1ss = of_property_read_bool(np, "brcm,enable-l1ss");
+ pcie->rcb_mps_mode = of_property_read_bool(np, "brcm,enable-mps-rcb");
+ of_property_read_u32(np, "brcm,tperst-clk-ms", &pcie->tperst_clk_ms);
pcie->rescal = devm_reset_control_get_optional_shared(&pdev->dev, "rescal");
if (IS_ERR(pcie->rescal))
@@ -1815,6 +2386,33 @@ static int brcm_pcie_probe(struct platfo
dev_err(pcie->dev, "probe of internal MSI failed");
goto fail;
}
+ } else if (pci_msi_enabled() && msi_np != pcie->np) {
+ /* Use RC_BAR1 for MIP access */
+ u64 msi_pci_addr;
+ u64 msi_phys_addr;
+
+ if (of_property_read_u64(msi_np, "brcm,msi-pci-addr", &msi_pci_addr)) {
+ dev_err(pcie->dev, "Unable to find MSI PCI address\n");
+ ret = -EINVAL;
+ goto fail;
+ }
+
+ if (of_property_read_u64(msi_np, "reg", &msi_phys_addr)) {
+ dev_err(pcie->dev, "Unable to find MSI physical address\n");
+ ret = -EINVAL;
+ goto fail;
+ }
+
+ writel(lower_32_bits(msi_pci_addr) | brcm_pcie_encode_ibar_size(0x1000),
+ pcie->base + PCIE_MISC_RC_BAR1_CONFIG_LO);
+ writel(upper_32_bits(msi_pci_addr),
+ pcie->base + PCIE_MISC_RC_BAR1_CONFIG_HI);
+
+ writel(lower_32_bits(msi_phys_addr) |
+ PCIE_MISC_UBUS_BAR1_CONFIG_REMAP_ACCESS_ENABLE_MASK,
+ pcie->base + PCIE_MISC_UBUS_BAR1_CONFIG_REMAP);
+ writel(upper_32_bits(msi_phys_addr),
+ pcie->base + PCIE_MISC_UBUS_BAR1_CONFIG_REMAP_HI);
}
bridge->ops = pcie->soc_base == BCM7425 ? &brcm7425_pcie_ops : &brcm_pcie_ops;
Reference in a new issue View git blame Copy permalink