151 		if (!ntb_topo_is_b2b(ndev->ntb.topo))  in ndev_reset_unsafe_flags()
192 		dev_dbg(&ndev->ntb.pdev->dev, "Peer db addr %llx\n", *db_addr);  in ndev_db_addr()
197 		dev_dbg(&ndev->ntb.pdev->dev, "Peer db size %llx\n", *db_size);  in ndev_db_addr()
288 		dev_dbg(&ndev->ntb.pdev->dev, "Peer spad addr %llx\n",  in ndev_spad_addr()
330 	dev_dbg(&ndev->ntb.pdev->dev, "vec %d vec_mask %llx\n", vec, vec_mask);  in ndev_interrupt()
336 			ntb_link_event(&ndev->ntb);  in ndev_interrupt()
340 		ntb_db_event(&ndev->ntb, vec);  in ndev_interrupt()
349 	dev_dbg(&nvec->ndev->ntb.pdev->dev, "irq: %d  nvec->num: %d\n",  in ndev_vec_isr()
359 	return ndev_interrupt(ndev, irq - ndev->ntb.pdev->irq);  in ndev_irq_isr()
369 	pdev = ndev->ntb.pdev;  in ndev_init_isr()
468 	pdev = ndev->ntb.pdev;  in ndev_deinit_isr()
502 	pdev = ndev->ntb.pdev;  in ndev_ntb_debugfs_read()
518 			 ntb_topo_string(ndev->ntb.topo));  in ndev_ntb_debugfs_read()
650 		if (ntb_topo_is_b2b(ndev->ntb.topo)) {  in ndev_ntb_debugfs_read()
761 	if (pdev_is_gen1(ndev->ntb.pdev))  in ndev_debugfs_read()
763 	else if (pdev_is_gen3(ndev->ntb.pdev))  in ndev_debugfs_read()
776 			debugfs_create_dir(pci_name(ndev->ntb.pdev),  in ndev_init_debugfs()
793 int intel_ntb_mw_count(struct ntb_dev *ntb, int pidx)  in intel_ntb_mw_count()  argument
798 	return ntb_ndev(ntb)->mw_count;  in intel_ntb_mw_count()
801 int intel_ntb_mw_get_align(struct ntb_dev *ntb, int pidx, int idx,  in intel_ntb_mw_get_align()  argument
806 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_mw_get_align()
820 	bar_size = pci_resource_len(ndev->ntb.pdev, bar);  in intel_ntb_mw_get_align()
828 		*addr_align = pci_resource_len(ndev->ntb.pdev, bar);  in intel_ntb_mw_get_align()
839 static int intel_ntb_mw_set_trans(struct ntb_dev *ntb, int pidx, int idx,  in intel_ntb_mw_set_trans()  argument
842 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_mw_set_trans()
859 	bar_size = pci_resource_len(ndev->ntb.pdev, bar);  in intel_ntb_mw_set_trans()
940 u64 intel_ntb_link_is_up(struct ntb_dev *ntb, enum ntb_speed *speed,  in intel_ntb_link_is_up()  argument
943 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_link_is_up()
962 static int intel_ntb_link_enable(struct ntb_dev *ntb,  in intel_ntb_link_enable()  argument
969 	ndev = container_of(ntb, struct intel_ntb_dev, ntb);  in intel_ntb_link_enable()
971 	if (ndev->ntb.topo == NTB_TOPO_SEC)  in intel_ntb_link_enable()
974 	dev_dbg(&ntb->pdev->dev,  in intel_ntb_link_enable()
978 		dev_dbg(&ntb->pdev->dev, "ignoring max_speed %d\n", max_speed);  in intel_ntb_link_enable()
980 		dev_dbg(&ntb->pdev->dev, "ignoring max_width %d\n", max_width);  in intel_ntb_link_enable()
993 int intel_ntb_link_disable(struct ntb_dev *ntb)  in intel_ntb_link_disable()  argument
998 	ndev = container_of(ntb, struct intel_ntb_dev, ntb);  in intel_ntb_link_disable()
1000 	if (ndev->ntb.topo == NTB_TOPO_SEC)  in intel_ntb_link_disable()
1003 	dev_dbg(&ntb->pdev->dev, "Disabling link\n");  in intel_ntb_link_disable()
1017 int intel_ntb_peer_mw_count(struct ntb_dev *ntb)  in intel_ntb_peer_mw_count()  argument
1020 	return ntb_ndev(ntb)->mw_count;  in intel_ntb_peer_mw_count()
1023 int intel_ntb_peer_mw_get_addr(struct ntb_dev *ntb, int idx,  in intel_ntb_peer_mw_get_addr()  argument
1026 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_peer_mw_get_addr()
1037 		*base = pci_resource_start(ndev->ntb.pdev, bar) +  in intel_ntb_peer_mw_get_addr()
1041 		*size = pci_resource_len(ndev->ntb.pdev, bar) -  in intel_ntb_peer_mw_get_addr()
1047 static int intel_ntb_db_is_unsafe(struct ntb_dev *ntb)  in intel_ntb_db_is_unsafe()  argument
1049 	return ndev_ignore_unsafe(ntb_ndev(ntb), NTB_UNSAFE_DB);  in intel_ntb_db_is_unsafe()
1052 u64 intel_ntb_db_valid_mask(struct ntb_dev *ntb)  in intel_ntb_db_valid_mask()  argument
1054 	return ntb_ndev(ntb)->db_valid_mask;  in intel_ntb_db_valid_mask()
1057 int intel_ntb_db_vector_count(struct ntb_dev *ntb)  in intel_ntb_db_vector_count()  argument
1061 	ndev = container_of(ntb, struct intel_ntb_dev, ntb);  in intel_ntb_db_vector_count()
1066 u64 intel_ntb_db_vector_mask(struct ntb_dev *ntb, int db_vector)  in intel_ntb_db_vector_mask()  argument
1068 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_db_vector_mask()
1076 static u64 intel_ntb_db_read(struct ntb_dev *ntb)  in intel_ntb_db_read()  argument
1078 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_db_read()
1085 static int intel_ntb_db_clear(struct ntb_dev *ntb, u64 db_bits)  in intel_ntb_db_clear()  argument
1087 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_db_clear()
1094 int intel_ntb_db_set_mask(struct ntb_dev *ntb, u64 db_bits)  in intel_ntb_db_set_mask()  argument
1096 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_db_set_mask()
1103 int intel_ntb_db_clear_mask(struct ntb_dev *ntb, u64 db_bits)  in intel_ntb_db_clear_mask()  argument
1105 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_db_clear_mask()
1112 static int intel_ntb_peer_db_addr(struct ntb_dev *ntb, phys_addr_t *db_addr,  in intel_ntb_peer_db_addr()  argument
1116 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_peer_db_addr()
1123 	if (unlikely(db_bits & ~ntb_ndev(ntb)->db_valid_mask))  in intel_ntb_peer_db_addr()
1136 static int intel_ntb_peer_db_set(struct ntb_dev *ntb, u64 db_bits)  in intel_ntb_peer_db_set()  argument
1138 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_peer_db_set()
1145 int intel_ntb_spad_is_unsafe(struct ntb_dev *ntb)  in intel_ntb_spad_is_unsafe()  argument
1147 	return ndev_ignore_unsafe(ntb_ndev(ntb), NTB_UNSAFE_SPAD);  in intel_ntb_spad_is_unsafe()
1150 int intel_ntb_spad_count(struct ntb_dev *ntb)  in intel_ntb_spad_count()  argument
1154 	ndev = container_of(ntb, struct intel_ntb_dev, ntb);  in intel_ntb_spad_count()
1159 u32 intel_ntb_spad_read(struct ntb_dev *ntb, int idx)  in intel_ntb_spad_read()  argument
1161 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_spad_read()
1168 int intel_ntb_spad_write(struct ntb_dev *ntb, int idx, u32 val)  in intel_ntb_spad_write()  argument
1170 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_spad_write()
1177 int intel_ntb_peer_spad_addr(struct ntb_dev *ntb, int pidx, int sidx,  in intel_ntb_peer_spad_addr()  argument
1180 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_peer_spad_addr()
1186 u32 intel_ntb_peer_spad_read(struct ntb_dev *ntb, int pidx, int sidx)  in intel_ntb_peer_spad_read()  argument
1188 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_peer_spad_read()
1195 int intel_ntb_peer_spad_write(struct ntb_dev *ntb, int pidx, int sidx,  in intel_ntb_peer_spad_write()  argument
1198 	struct intel_ntb_dev *ndev = ntb_ndev(ntb);  in intel_ntb_peer_spad_write()
1224 	rc = pci_read_config_word(ndev->ntb.pdev,  in xeon_poll_link()
1239 	if (ndev->ntb.topo == NTB_TOPO_SEC)  in xeon_link_is_up()
1269 		dev_dbg(&ndev->ntb.pdev->dev, "PPD %d split bar\n", ppd);  in xeon_ppd_bar4_split()
1299 	pdev = ndev->ntb.pdev;  in xeon_setup_b2b_mw()
1313 		bar_size = pci_resource_len(ndev->ntb.pdev, b2b_bar);  in xeon_setup_b2b_mw()
1536 	struct device *dev = &ndev->ntb.pdev->dev;  in xeon_init_ntb()
1549 	switch (ndev->ntb.topo) {  in xeon_init_ntb()
1609 		if (ndev->ntb.topo == NTB_TOPO_B2B_USD) {  in xeon_init_ntb()
1646 	pdev = ndev->ntb.pdev;  in xeon_init_dev()
1721 	ndev->ntb.topo = xeon_ppd_topo(ndev, ppd);  in xeon_init_dev()
1723 		ntb_topo_string(ndev->ntb.topo));  in xeon_init_dev()
1724 	if (ndev->ntb.topo == NTB_TOPO_NONE)  in xeon_init_dev()
1727 	if (ndev->ntb.topo != NTB_TOPO_SEC) {  in xeon_init_dev()
1786 	rc = dma_coerce_mask_and_coherent(&ndev->ntb.dev,  in intel_ntb_init_pci()
1814 	struct pci_dev *pdev = ndev->ntb.pdev;  in intel_ntb_deinit_pci()
1829 	ndev->ntb.pdev = pdev;  in ndev_init_struct()
1830 	ndev->ntb.topo = NTB_TOPO_NONE;  in ndev_init_struct()
1831 	ndev->ntb.ops = &intel_ntb_ops;  in ndev_init_struct()
1887 		ndev->ntb.ops = &intel_ntb3_ops;  in intel_ntb_pci_probe()
1908 	rc = ntb_register_device(&ndev->ntb);  in intel_ntb_pci_probe()
1932 	ntb_unregister_device(&ndev->ntb);  in intel_ntb_pci_remove()