56  *             expiration of the in-service entity
172 	 * as it must never become an in-service entity.  in bfq_update_parent_budget()
186  * service, according to the restrictive definition of the field
188  * entity that is about to be set in service.
191  * next service according to the that definition, because entity is
192  * about to become the in-service queue. This function then returns
195  * In contrast, entity could still be a candidate for next service if
197  * even if one of its children is about to be set in service, other
200  * non-queue entity is not a candidate for next-service only if it has
216 	 * not account for the in-service entity in case the latter is  in bfq_no_longer_next_in_service()
252  * service allowed in one timestamp delta (small shift values increase it),
271  * bfq_delta - map service into the virtual time domain.
272  * @service: amount of service.
275 static u64 bfq_delta(unsigned long service, unsigned long weight)  in bfq_delta()  argument
277 	return div64_ul((u64)service << WFQ_SERVICE_SHIFT, weight);  in bfq_delta()
283  * @service: the service to be charged to the entity.
285 static void bfq_calc_finish(struct bfq_entity *entity, unsigned long service)  in bfq_calc_finish()  argument
290 		bfq_delta(service, entity->weight);  in bfq_calc_finish()
295 			service, entity->weight);  in bfq_calc_finish()
299 			bfq_delta(service, entity->weight));  in bfq_calc_finish()
335  * @st: the service tree of the owning @entity.
461  * @st: the service tree of the entity.
603  * @st: the service tree containing the tree.
626  * @st: the service tree.
628  * @is_in_service: true if entity is currently the in-service entity.
631  * a queue, and the latter is not in service, then release the service
633  * fact, in this case, there is really no more service reference to
634  * the queue, as the latter is also outside any service tree. If,
635  * instead, the queue is in service, then __bfq_bfqd_reset_in_service
653  * @st: service tree for the entity.
665  * @st: the service tree to act upon.
702  * entity implies changing the destination service trees for that
704  * of the service trees for its previous class, then the state of the
705  * entity would become more complex: none of the new possible service
707  * match any of the possible service trees on which the entity
813  *                   service.
818  * are synchronized every time a new bfqq is selected for service.  By now,
836 		entity->service += served;  in bfq_bfqq_served()
845  * bfq_bfqq_charge_time - charge an amount of service equivalent to the length
847  *			  service.
849  * @bfqq: the queue that needs a service update.
850  * @time_ms: the amount of time during which the queue has received service
853  * the queue with service fairness may impair throughput, more or less
855  * are provided with time fairness instead of service fairness. This
857  * engine works in the service, and not in the time domain. The trick
858  * is charging these queues with an inflated amount of service, equal
859  * to the amount of service that they would have received during their
860  * service slot if they had been fast, i.e., if their requests had
866  * during the service slot of a queue may be served after that service
868  * correlated with the duration of the service slot. This is
869  * especially true for short service slots.
879 	int tot_serv_to_charge = max(serv_to_charge_for_time, entity->service);  in bfq_bfqq_charge_time()
886 			max_t(int, 0, tot_serv_to_charge - entity->service));  in bfq_bfqq_charge_time()
896 	 * When this function is invoked, entity is not in any service  in bfq_update_fin_time_enqueue()
919 	 * time. This would simply break service guarantees.  in bfq_update_fin_time_enqueue()
992 		 * entity is about to be inserted into a service tree,  in __bfq_activate_entity()
993 		 * and then set in service: get a reference to make  in __bfq_activate_entity()
995 		 * longer in service or scheduled for service.  in __bfq_activate_entity()
1040 		 * We are requeueing the current in-service entity,  in __bfq_requeue_entity()
1043 		 * - entity represents the in-service queue, and the  in __bfq_requeue_entity()
1044 		 *   in-service queue is being requeued after an  in __bfq_requeue_entity()
1055 		 * service that the entity has received while in  in __bfq_requeue_entity()
1056 		 * service. This is done by the next instructions. The  in __bfq_requeue_entity()
1061 		bfq_calc_finish(entity, entity->service);  in __bfq_requeue_entity()
1065 		 * when set in service, then it was not extracted from  in __bfq_requeue_entity()
1078 	} else { /* The entity is already active, and not in service */  in __bfq_requeue_entity()
1107 		  * in service or already queued on the active tree,  in __bfq_activate_requeue_entity()
1113 		 * Not in service and not queued on its active tree:  in __bfq_activate_requeue_entity()
1131  *             of the in-service queue
1150  * __bfq_deactivate_entity - update sched_data and service trees for
1157  * entity may be on no tree if in service.
1180 	bfq_calc_finish(entity, entity->service);  in __bfq_deactivate_entity()
1186 		 * Non in-service entity: nobody will take care of  in __bfq_deactivate_entity()
1187 		 * resetting its service counter on expiration. Do it  in __bfq_deactivate_entity()
1190 		entity->service = 0;  in __bfq_deactivate_entity()
1210  *             of the in-service queue
1273 		 * deactivation, to preserve service guarantees, and  in bfq_deactivate_entity()
1284 	 * about to stop receiving service, or reposition them if this  in bfq_deactivate_entity()
1313  * @st: the service tree to act upon.
1338  * @st: the service tree to select from.
1377  * @st: the service tree.
1379  * If there is no in-service entity for the sched_data st belongs to,
1380  * then return the entity that will be set in service if:
1381  * 1) the parent entity this st belongs to is set in service;
1389  * In contrast, if there is an in-service entity, then return the
1390  * entity that would be set in service if not only the above
1392  * in-service entity, on expiration,
1413 	 * If there is no in-service entity for the sched_data this  in __bfq_lookup_next_entity()
1416 	 * eligible. If, instead, there is an in-service entity, then  in __bfq_lookup_next_entity()
1418 	 * eligible entity, namely the in-service one (even if the  in __bfq_lookup_next_entity()
1420 	 * service).  in __bfq_lookup_next_entity()
1433  * @expiration: true if we are on the expiration path of the in-service queue
1470 		 * of the in-service queue. In this case, even if  in bfq_lookup_next_entity()
1473 		 * in service any more, and, if needed, has already  in bfq_lookup_next_entity()
1504  * Get next queue for service.
1517 	 * serve. Set in service all the entities visited along the  in bfq_get_next_queue()
1523 		 * WARNING. We are about to set the in-service entity  in bfq_get_next_queue()
1527 		 * service order in sd changed as a consequence of the  in bfq_get_next_queue()
1540 		 * service of the sd->next_in_service entity  in bfq_get_next_queue()
1552 		 * service, then it must be extracted from its active  in bfq_get_next_queue()
1575 		 * in service. In fact, only after we know which is  in bfq_get_next_queue()
1586 	 * path from the leaf entity just set in service to the root.  in bfq_get_next_queue()
1598 /* returns true if the in-service queue gets freed */
1610 	 * When this function is called, all in-service entities have  in __bfq_bfqd_reset_in_service()
1619 	 * in_serv_entity is no longer in service, so, if it is in no  in __bfq_bfqd_reset_in_service()
1620 	 * service tree either, then release the service reference to  in __bfq_bfqd_reset_in_service()
1626 		 * longer, then the service reference may be the only  in __bfq_bfqd_reset_in_service()
1667  * the service tree. As a special case, it can be invoked during an