Lines Matching full:we
91 /* We need _PAGE_PRESENT and _PAGE_ACCESSED set */
93 /* We do the user/kernel test for the PID here along with the RW test
95 /* We pre-test some combination of permissions to avoid double
98 * We move the ESR:ST bit into the position of _PAGE_BAP_SW in the PTE
103 * writeable, we will take a new fault later, but that should be
106 * We also move ESR_ST in _PAGE_DIRTY position
109 * MAS1 is preset for all we need except for TID that needs to
137 * We are entered with:
176 /* Now we build the MAS:
219 /* We need to check if it was an instruction miss */
239 /* We do the user/kernel test for the PID here along with the RW test
253 * No HES or NV hint on TLB1, so we need to do software round-robin
254 * No tlbsrx. so we need a spinlock, and we have to deal
281 * We are entered with:
296 * Search if we already have an indirect entry for that virtual
297 * address, and if we do, bail out.
324 * Erratum A-008139 says that we can't use tlbwe to change
379 /* Now, we need to walk the page tables. First check if we are in
407 /* Now we build the MAS for a 2M indirect page:
460 * Now we build the MAS for a huge page.
504 /* We need to check if it was an instruction miss */
525 /* Now we handle the fault proper. We only save DEAR in normal
527 * We could probably also optimize by not saving SRR0/1 in the
539 * we don't know whether we are trying to fault in a first level
540 * virtual address or a virtual page table address. We can get that
541 * from bit 0x1 of the region ID which we have set for a page table
549 /* We need _PAGE_PRESENT and _PAGE_ACCESSED set */
553 /* We do the user/kernel test for the PID here along with the RW test
557 /* We pre-test some combination of permissions to avoid double
560 * We move the ESR:ST bit into the position of _PAGE_BAP_SW in the PTE
565 * writeable, we will take a new fault later, but that should be
568 * We also move ESR_ST in _PAGE_DIRTY position
571 * MAS1 is preset for all we need except for TID that needs to
583 /* We got a crappy address, just fault with whatever DEAR and ESR
593 /* If we take a recursive fault, the second level handler may need
594 * to know whether we are handling a data or instruction fault in
595 * order to get to the right store fault handler. We provide that
600 /* Now we handle the fault proper. We only save DEAR in the non
601 * linear mapping case since we know the linear mapping case will
602 * not re-enter. We could indeed optimize and also not save SRR0/1
613 /* We do the user/kernel test for the PID here along with the RW test
630 /* We got a crappy address, just fault */
636 * misses. We are entered with:
653 /* So we first construct the page table address. We do that by
658 * NOTE: For 64K pages, we do things slightly differently in
675 /* Now we build the MAS:
719 /* We don't bother with restoring DEAR or ESR since we know we are
727 /* We need to check if it was an instruction miss */
742 * misses. We are entered with:
754 * That means we can always get the original fault DEAR at
759 * 0 so we don't care too much about clobbers
761 * XXX That code was written back when we couldn't clobber r14. We can now,
762 * so we could probably optimize things a bit
765 /* Are we hitting a kernel page table ? */
770 * and we happen to have the swapper_pg_dir at offset 8 from the user
775 /* If kernel, we need to clear MAS1 TID */
792 /* Now, we need to walk the page tables. First check if we are in
825 /* Ok, we're all right, we can now create a kernel translation for
828 /* Now we build the MAS:
837 * So we only do MAS 2 and 3 for now...
853 /* If we fault here, things are a little bit tricky. We need to call
854 * either data or instruction store fault, and we need to retrieve
857 * The thing is, we know that in normal circumstances, this is
859 * level TLB miss for HW load, so we should be able to peek at the
862 * However, we do need to double check that, because we may just hit
864 * areas. If that is the case, we do a data fault. (We can't get here
867 * Note also that when going to a fault, we must unwind the previous
868 * level as well. Since we are doing that, we don't need to clear or
875 /* We dig the original DEAR and ESR from slot 0 */
879 /* We check for the "special" ESR value for instruction faults */
891 * not have been clobbered, let's just fault with what we have
908 /* Now we handle the fault proper. We only save DEAR in normal
910 * We could probably also optimize by not saving SRR0/1 in the
921 /* We do the user/kernel test for the PID here along with the RW test
934 /* We got a crappy address, just fault with whatever DEAR and ESR
944 /* If we take a recursive fault, the second level handler may need
945 * to know whether we are handling a data or instruction fault in
946 * order to get to the right store fault handler. We provide that
951 /* Now we handle the fault proper. We only save DEAR in the non
952 * linear mapping case since we know the linear mapping case will
953 * not re-enter. We could indeed optimize and also not save SRR0/1
964 /* We do the user/kernel test for the PID here along with the RW test
977 /* We got a crappy address, just fault */
984 * misses. We are entered with:
1003 /* Search if we already have a TLB entry for that virtual address, and
1004 * if we do, bail out.
1011 /* Now, we need to walk the page tables. First check if we are in
1042 /* Ok, we're all right, we can now create an indirect entry for
1045 * The last trick is now that because we use "half" pages for
1046 * the HTW (1M IND is 2K and 256M IND is 32K) we need to account
1048 * problem as we already use 32K arrays (half PTE pages), but for
1049 * 4K page we need to extract a bit from the virtual address and
1053 /* Now we build the MAS:
1071 /* We don't bother with restoring DEAR or ESR since we know we are
1079 /* We need to check if it was an instruction miss. We know this
1093 * mapping misses. We are entered with:
1104 * In addition we know that we will not re-enter, so in theory, we could
1105 * use a simpler epilog not restoring SRR0/1 etc.. but we'll do that later.
1107 * We also need to be careful about MAS registers here & TLB reservation,
1108 * as we know we'll have clobbered them if we interrupt the main TLB miss
1109 * handlers in which case we probably want to do a full restart at level
1112 * Note: If we care about performance of that core, we can easily shuffle
1116 /* For now, we assume the linear mapping is contiguous and stops at
1117 * linear_map_top. We also assume the size is a multiple of 1G, thus
1118 * we only use 1G pages for now. That might have to be changed in a
1137 /* Now we build the remaining MAS. MAS0 and 2 should be fine
1139 * mapping is linear, so we just take the address, clear the
1154 /* We use the "error" epilog for success as we do want to
1156 * We do that because we can't resume a fault within a TLB
1163 /* We keep the DEAR and ESR around, this shouldn't have happened */