Lines Matching full:we
95 /* We need _PAGE_PRESENT and _PAGE_ACCESSED set */
97 /* We do the user/kernel test for the PID here along with the RW test
99 /* We pre-test some combination of permissions to avoid double
102 * We move the ESR:ST bit into the position of _PAGE_BAP_SW in the PTE
107 * writeable, we will take a new fault later, but that should be
110 * We also move ESR_ST in _PAGE_DIRTY position
113 * MAS1 is preset for all we need except for TID that needs to
134 * We are entered with:
182 /* Now we build the MAS:
224 /* We need to check if it was an instruction miss */
244 /* We do the user/kernel test for the PID here along with the RW test
259 * No HES or NV hint on TLB1, so we need to do software round-robin
260 * No tlbsrx. so we need a spinlock, and we have to deal
287 * We are entered with:
302 * Search if we already have an indirect entry for that virtual
303 * address, and if we do, bail out.
330 * Erratum A-008139 says that we can't use tlbwe to change
385 /* Now, we need to walk the page tables. First check if we are in
413 /* Now we build the MAS for a 2M indirect page:
466 * Now we build the MAS for a huge page.
508 /* We need to check if it was an instruction miss */
530 /* Now we handle the fault proper. We only save DEAR in normal
532 * We could probably also optimize by not saving SRR0/1 in the
542 * we don't know whether we are trying to fault in a first level
543 * virtual address or a virtual page table address. We can get that
544 * from bit 0x1 of the region ID which we have set for a page table
552 /* We need _PAGE_PRESENT and _PAGE_ACCESSED set */
556 /* We do the user/kernel test for the PID here along with the RW test
560 /* We pre-test some combination of permissions to avoid double
563 * We move the ESR:ST bit into the position of _PAGE_BAP_SW in the PTE
568 * writeable, we will take a new fault later, but that should be
571 * We also move ESR_ST in _PAGE_DIRTY position
574 * MAS1 is preset for all we need except for TID that needs to
587 /* We got a crappy address, just fault with whatever DEAR and ESR
597 /* If we take a recursive fault, the second level handler may need
598 * to know whether we are handling a data or instruction fault in
599 * order to get to the right store fault handler. We provide that
604 /* Now we handle the fault proper. We only save DEAR in the non
605 * linear mapping case since we know the linear mapping case will
606 * not re-enter. We could indeed optimize and also not save SRR0/1
615 /* We do the user/kernel test for the PID here along with the RW test
633 /* We got a crappy address, just fault */
639 * misses. We are entered with:
650 /* So we first construct the page table address. We do that by
655 * NOTE: For 64K pages, we do things slightly differently in
680 /* Now we build the MAS:
728 /* We don't bother with restoring DEAR or ESR since we know we are
736 /* We need to check if it was an instruction miss */
751 * misses. We are entered with:
763 * That means we can always get the original fault DEAR at
768 * 0 so we don't care too much about clobbers
770 * XXX That code was written back when we couldn't clobber r14. We can now,
771 * so we could probably optimize things a bit
774 /* Are we hitting a kernel page table ? */
778 * and we happen to have the swapper_pg_dir at offset 8 from the user
783 /* If kernel, we need to clear MAS1 TID */
791 /* Search if we already have a TLB entry for that virtual address, and
792 * if we do, bail out.
798 /* Now, we need to walk the page tables. First check if we are in
830 /* Ok, we're all right, we can now create a kernel translation for
833 /* Now we build the MAS:
842 * So we only do MAS 2 and 3 for now...
860 /* We have overridden MAS2:EPN but currently our primary TLB miss
862 * if we ever optimize the primary handler to not write MAS2 on
863 * some cases, we'll have to restore MAS2:EPN here based on the
864 * original fault's DEAR. If we do that we have to modify the
868 * However, one nasty thing we did is we cleared the reservation
869 * (well, potentially we did). We do a trick here thus if we
870 * are not a level 0 exception (we interrupted the TLB miss) we
887 /* If we fault here, things are a little bit tricky. We need to call
888 * either data or instruction store fault, and we need to retrieve
891 * The thing is, we know that in normal circumstances, this is
893 * level TLB miss for HW load, so we should be able to peek at the
896 * However, we do need to double check that, because we may just hit
898 * areas. If that is the case, we do a data fault. (We can't get here
901 * Note also that when going to a fault, we must unwind the previous
902 * level as well. Since we are doing that, we don't need to clear or
909 /* We dig the original DEAR and ESR from slot 0 */
913 /* We check for the "special" ESR value for instruction faults */
925 * not have been clobbered, let's just fault with what we have
942 /* Now we handle the fault proper. We only save DEAR in normal
944 * We could probably also optimize by not saving SRR0/1 in the
953 /* We do the user/kernel test for the PID here along with the RW test
967 /* We got a crappy address, just fault with whatever DEAR and ESR
977 /* If we take a recursive fault, the second level handler may need
978 * to know whether we are handling a data or instruction fault in
979 * order to get to the right store fault handler. We provide that
984 /* Now we handle the fault proper. We only save DEAR in the non
985 * linear mapping case since we know the linear mapping case will
986 * not re-enter. We could indeed optimize and also not save SRR0/1
995 /* We do the user/kernel test for the PID here along with the RW test
1009 /* We got a crappy address, just fault */
1016 * misses. We are entered with:
1030 /* Search if we already have a TLB entry for that virtual address, and
1031 * if we do, bail out.
1038 /* Now, we need to walk the page tables. First check if we are in
1069 /* Ok, we're all right, we can now create an indirect entry for
1072 * The last trick is now that because we use "half" pages for
1073 * the HTW (1M IND is 2K and 256M IND is 32K) we need to account
1075 * problem as we already use 32K arrays (half PTE pages), but for
1076 * 4K page we need to extract a bit from the virtual address and
1080 /* Now we build the MAS:
1102 /* We don't bother with restoring DEAR or ESR since we know we are
1110 /* We need to check if it was an instruction miss. We know this
1124 * mapping misses. We are entered with:
1135 * In addition we know that we will not re-enter, so in theory, we could
1136 * use a simpler epilog not restoring SRR0/1 etc.. but we'll do that later.
1138 * We also need to be careful about MAS registers here & TLB reservation,
1139 * as we know we'll have clobbered them if we interrupt the main TLB miss
1140 * handlers in which case we probably want to do a full restart at level
1143 * Note: If we care about performance of that core, we can easily shuffle
1147 /* For now, we assume the linear mapping is contiguous and stops at
1148 * linear_map_top. We also assume the size is a multiple of 1G, thus
1149 * we only use 1G pages for now. That might have to be changed in a
1168 /* Now we build the remaining MAS. MAS0 and 2 should be fine
1170 * mapping is linear, so we just take the address, clear the
1189 /* We use the "error" epilog for success as we do want to
1191 * We do that because we can't resume a fault within a TLB
1198 /* We keep the DEAR and ESR around, this shouldn't have happened */