56  *             expiration of the in-service entity
162 	 * as it must never become an in-service entity.  in bfq_update_parent_budget()
176  * service, according to the restrictive definition of the field
178  * entity that is about to be set in service.
181  * next service according to the that definition, because entity is
182  * about to become the in-service queue. This function then returns
185  * In contrast, entity could still be a candidate for next service if
187  * even if one of its children is about to be set in service, other
190  * non-queue entity is not a candidate for next-service only if it has
206 	 * not account for the in-service entity in case the latter is  in bfq_no_longer_next_in_service()
263  * service allowed in one timestamp delta (small shift values increase it),
282  * bfq_delta - map service into the virtual time domain.
283  * @service: amount of service.
286 static u64 bfq_delta(unsigned long service, unsigned long weight)  in bfq_delta()  argument
288 	return div64_ul((u64)service << WFQ_SERVICE_SHIFT, weight);  in bfq_delta()
294  * @service: the service to be charged to the entity.
296 static void bfq_calc_finish(struct bfq_entity *entity, unsigned long service)  in bfq_calc_finish()  argument
301 		bfq_delta(service, entity->weight);  in bfq_calc_finish()
306 			service, entity->weight);  in bfq_calc_finish()
310 			bfq_delta(service, entity->weight));  in bfq_calc_finish()
346  * @st: the service tree of the owning @entity.
472  * @st: the service tree of the entity.
589  * @st: the service tree containing the tree.
612  * @st: the service tree.
614  * @is_in_service: true if entity is currently the in-service entity.
617  * a queue, and the latter is not in service, then release the service
619  * fact, in this case, there is really no more service reference to
620  * the queue, as the latter is also outside any service tree. If,
621  * instead, the queue is in service, then __bfq_bfqd_reset_in_service
639  * @st: service tree for the entity.
651  * @st: the service tree to act upon.
688  * entity implies changing the destination service trees for that
690  * of the service trees for its previous class, then the state of the
691  * entity would become more complex: none of the new possible service
693  * match any of the possible service trees on which the entity
779  *                   service.
784  * are synchronized every time a new bfqq is selected for service.  By now,
802 		entity->service += served;  in bfq_bfqq_served()
811  * bfq_bfqq_charge_time - charge an amount of service equivalent to the length
813  *			  service.
815  * @bfqq: the queue that needs a service update.
816  * @time_ms: the amount of time during which the queue has received service
819  * the queue with service fairness may impair throughput, more or less
821  * are provided with time fairness instead of service fairness. This
823  * engine works in the service, and not in the time domain. The trick
824  * is charging these queues with an inflated amount of service, equal
825  * to the amount of service that they would have received during their
826  * service slot if they had been fast, i.e., if their requests had
832  * during the service slot of a queue may be served after that service
834  * correlated with the duration of the service slot. This is
835  * especially true for short service slots.
845 	int tot_serv_to_charge = max(serv_to_charge_for_time, entity->service);  in bfq_bfqq_charge_time()
852 			max_t(int, 0, tot_serv_to_charge - entity->service));  in bfq_bfqq_charge_time()
862 	 * When this function is invoked, entity is not in any service  in bfq_update_fin_time_enqueue()
885 	 * time. This would simply break service guarantees.  in bfq_update_fin_time_enqueue()
958 		 * entity is about to be inserted into a service tree,  in __bfq_activate_entity()
959 		 * and then set in service: get a reference to make  in __bfq_activate_entity()
961 		 * longer in service or scheduled for service.  in __bfq_activate_entity()
993 		 * We are requeueing the current in-service entity,  in __bfq_requeue_entity()
996 		 * - entity represents the in-service queue, and the  in __bfq_requeue_entity()
997 		 *   in-service queue is being requeued after an  in __bfq_requeue_entity()
1008 		 * service that the entity has received while in  in __bfq_requeue_entity()
1009 		 * service. This is done by the next instructions. The  in __bfq_requeue_entity()
1014 		bfq_calc_finish(entity, entity->service);  in __bfq_requeue_entity()
1018 		 * when set in service, then it was not extracted from  in __bfq_requeue_entity()
1031 	} else { /* The entity is already active, and not in service */  in __bfq_requeue_entity()
1060 		  * in service or already queued on the active tree,  in __bfq_activate_requeue_entity()
1066 		 * Not in service and not queued on its active tree:  in __bfq_activate_requeue_entity()
1084  *             of the in-service queue
1099  * __bfq_deactivate_entity - update sched_data and service trees for
1106  * entity may be on no tree if in service.
1129 	bfq_calc_finish(entity, entity->service);  in __bfq_deactivate_entity()
1135 		 * Non in-service entity: nobody will take care of  in __bfq_deactivate_entity()
1136 		 * resetting its service counter on expiration. Do it  in __bfq_deactivate_entity()
1139 		entity->service = 0;  in __bfq_deactivate_entity()
1159  *             of the in-service queue
1222 		 * deactivation, to preserve service guarantees, and  in bfq_deactivate_entity()
1233 	 * about to stop receiving service, or reposition them if this  in bfq_deactivate_entity()
1262  * @st: the service tree to act upon.
1287  * @st: the service tree to select from.
1326  * @st: the service tree.
1327  * @in_service: whether or not there is an in-service entity for the sched_data
1330  * If there is no in-service entity for the sched_data st belongs to,
1331  * then return the entity that will be set in service if:
1332  * 1) the parent entity this st belongs to is set in service;
1340  * In contrast, if there is an in-service entity, then return the
1341  * entity that would be set in service if not only the above
1343  * in-service entity, on expiration,
1364 	 * If there is no in-service entity for the sched_data this  in __bfq_lookup_next_entity()
1367 	 * eligible. If, instead, there is an in-service entity, then  in __bfq_lookup_next_entity()
1369 	 * eligible entity, namely the in-service one (even if the  in __bfq_lookup_next_entity()
1371 	 * service).  in __bfq_lookup_next_entity()
1384  * @expiration: true if we are on the expiration path of the in-service queue
1421 		 * of the in-service queue. In this case, even if  in bfq_lookup_next_entity()
1424 		 * in service any more, and, if needed, has already  in bfq_lookup_next_entity()
1452  * Get next queue for service.
1465 	 * serve. Set in service all the entities visited along the  in bfq_get_next_queue()
1471 		 * WARNING. We are about to set the in-service entity  in bfq_get_next_queue()
1475 		 * service order in sd changed as a consequence of the  in bfq_get_next_queue()
1488 		 * service of the sd->next_in_service entity  in bfq_get_next_queue()
1500 		 * service, then it must be extracted from its active  in bfq_get_next_queue()
1523 		 * in service. In fact, only after we know which is  in bfq_get_next_queue()
1534 	 * path from the leaf entity just set in service to the root.  in bfq_get_next_queue()
1546 /* returns true if the in-service queue gets freed */
1558 	 * When this function is called, all in-service entities have  in __bfq_bfqd_reset_in_service()
1567 	 * in_serv_entity is no longer in service, so, if it is in no  in __bfq_bfqd_reset_in_service()
1568 	 * service tree either, then release the service reference to  in __bfq_bfqd_reset_in_service()
1574 		 * longer, then the service reference may be the only  in __bfq_bfqd_reset_in_service()
1641  * the service tree. As a special case, it can be invoked during an