Row Allocation Negotiation for a Fleet of Strawberry Harvesting Robots

High production costs, increasing populations, and labor shortages have driven the demand for automation in the agricultural industry [1–3]. Many operations in agricultural fields, such as weed control [4] and harvesting [5], have been or are being automated. Specialty crops, such as strawberries, are a research and engineering focal point of automation [6]. However, it is challenging to use robotics in some strawberry field operations which require more control and dexterity, such as harvesting, than grains or field crops [7]. To date, most strawberries are manually harvested, accounting for over 25% of the total production costs [7,8]. In Florida, many strawberry growers reported labor shortages while crops are in season [8], resulting in some of the yield being abandoned. Although autonomous harvesting has shown promising results in greenhouse conditions [9–11], advancements in field settings are quite limited. To increase the harvesting efficiency, industry players such as Agrobot² and Harvest Croo³ developed robotic platforms with multiple independent harvesters that span the width of several strawberry rows. However, these approaches have foreseen inadequacies in the long run such as transportation difficulty, prone to single-point-of-failure, significant downtime impact, low platform flexibility, and low adaptation to field variations. It becomes natural to utilize cooperative, small robotic platforms in strawberry harvesting, analogous to a group of human pickers. Nevertheless, multirobot systems must address additional technique challenges, one of which is effectively scheduling a robot to a row in real-time.

So far, all research in scheduling can be broadly categorized as centralized or decentralized. To the best of our knowledge, centralized approaches are popular in solving robotic platform scheduling problems in agricultural fields, mainly due to the possibility that not many robots, if multiple robots are present, would work in one field at the same time [12–18]. For example, an automatic system for vineyards [12] utilizes a central computer to assign paths for robots to irrigate high-priority vines. Restricted by the layout of vineyards, robots can only change rows at row headlines [12,16]. In the case of an herbicide-spraying robot, task sets are assigned from a central computer that generates random solutions, improving at each iteration and converging on a set of assignments [13]. Using a cost matrix to assign tasks in a centralized approach can account for external factors like crop rotation, weather constraints, and urgency to harvest [14]. Agricultural surveillance drones can have flight paths calculated and assigned via a centralized computer [18], sometimes assisted by negotiations among agents [17].

Decentralized approaches such as bundle algorithms, greedy algorithms, and auction algorithms are widely seen in other application domains [19–21]. Choi et al. [20] proposed a two-layer decentralized algorithm to optimally assign multiple tasks to each agent with applications in office cleaning and building emergency surveying [19]. In those studies [19,20], each agent generates a bundle list via an auction algorithm in Phase 1, and conflicts are resolved via a consensus approach in Phase 2. Auction algorithms involve a bidding process where tasks are assigned based on the associated values [21], and the process is overseen by a centralized computer to reduce communication time and achieve consensus.

In this paper, a tri-layered negotiation algorithm is proposed for real-time scheduling consisting of a decentralized bidding process between neighboring robots (layer 1 and layer 2) and, if needed, a centralized algorithm afterward (layer 3). In layer 1, robots determine if they are able to continue harvesting. Layer 2 contains two algorithms: one for robots bidding to help neighbors and another for robots overseeing the bidding process initiated by its neighbors. If a solution cannot be obtained in the decentralized layer, the Hungarian Algorithm [22] is utilized in layer 3 to assign tasks to each robot requesting a new assignment. Robots are incentivized to cooperate like human pickers to harvest the field in minimal time while collecting as many strawberries as they can. However the robots are subjected to motion limitations due to row size and their own capabilities.

The work in this paper proposes a row allocation negotiation algorithm to address an important gap in strawberry harvesting via cooperative, small harvesting robots. The technical contributions of the study are summarized here. (i) The decentralized part of the algorithm is scalable. The complexity of the centralized part is O(N³), but the probability of using it is low. (ii) The computational time is significantly reduced as compared to centralized approaches. (iii) The algorithm is easy to implement.

The remainder of the paper is organized as follows. In Sec. 2, the row allocation problem for robots in a strawberry field is defined with a list of assumptions. The algorithm is discussed in Sec. 3, and its communication, convergence, and computational complexity analyses are given in Sec. 4. The proposed approach is validated in a Monte Carlo simulation and its performance is discussed in Sec. 5. Conclusions are given in Sec. 6.

There are a total of N_s rows in a strawberry field. For a typical field, two lines of trees are planted in every row, with mostly uniform distances between trees. There are N_hr harvesting robots in the fleet, each with identical functionalities and physical characteristics.

Figure 1 depicts a potential configuration of robots in a field, in which robots, their velocities, and the rows they occupy are shown along with descriptive information about the dimensions of the rows and the sets they belong to.

The following assumptions and constraints are considered in the scheduling algorithm development.

The optimization is subject to the constraints implied in the aforementioned assumptions. The nearest-neighbor topology is chosen for the communication considering typical commercial strawberry farm size, off-the-shelf wireless communication range, and number of harvesting robots in a fleet. Robot i, i = 1, …, N_hr, can only communicate with its neighboring robots in row(i⁺) and row(i⁻), as well as the truck when necessary. The maximum number of bid iterations n_req among neighboring robots is set to two, but can be adjusted by users.

The algorithm is divided into three decision layers comprised four algorithms. The first layer’s algorithm executes every time a robot finishes harvesting a distance equal to its body length l_i. If robot i receives a request from its neighbor in row(j) ∈ [row(i)⁺, row(i)⁻], it will not process the request until it finishes its current harvesting distance (e.g., l_i = 1m).

The algorithm begins in layer 1 (the pseudo code in Table 1), which is triggered when robot i harvests a distance l_i and reads in information from its file to decide if it should continue harvesting, etc. (line 1 of Table 1). Data in the file, updated by robot i itself, its neighboring robots, and/or the collection trucks, contain information about the farm, the progress, and condition of the robot fleet, and the inquiries sent by its neighboring robots. A robot will check for conditions that render it unable to harvest (lines 2 and 3 of Table 1). If robot i is at the end of its row, i.e., $λi=100%$ (line 4 of Table 1), it delivers its basket to the collection truck and returns to its row (Assumption 5). Robot i will determine which row to harvest next (lines 6–8 of Table 1) by either going to a new nearby row or entering Algorithm 2 to offer help to neighboring robots. Line 9 of Table 1 comes directly from the basket capacity definition in Assumption 4. If a robot receives an offer for a neighboring robot to come help (line 11 of Table 1), it enters Algorithm 3 to determine if this offer will be accepted. Otherwise, robot i will continue harvesting (line 14 of Table 1, Assumption 3).

In Algorithm 2, robot i initiates the bidding process by sending time incentives to its neighboring robots. Time incentives are used as currency in an auction process, where smaller time incentives are more valuable. The neighboring robots that receive time incentives will oversee the auction and determine the appropriate outcomes. We define n_req as the number of times robot i offers to help its neighbor. Robot i offers to help one of its neighboring robots in row(j) ∈ [row(i)⁺, row(i)⁻] and waits long enough for robot j to respond (e.g., t_c ranging between 1 and 3 s) (line 5 of Table 2). The time incentive used in Algorithm 2 is calculated as follows. Per Assumption 5, a robot has an empty basket when calculating time incentives, therefore it does not need to factor in basket delivery times. The remaining fraction of row(j) that has not been harvested yet is defined as L_j (1 − λ_j), where L_j is the length of row(j) and λ_j is the percentage of the overbed length harvested. The distance robot i covers in transit is the distance between robot i and the end of row(j), d_i,j (Assumption 6), plus L_j(1 − λ_j)/2. Furthermore, the time needed for both robot i and robot j to finish harvesting the row where robot j is in is counted in the time incentive. This yields the time incentive for robot i to help robot j, $tij=[di,j+Lj(1−λj)/2]/vi,nh+Lj(1−λj)/(vi,h+vj,h)$⁠, consistent with Assumption 9. If robot i receives positive responses (indicating it has won the bid) from both neighbors, the neighbor corresponding to the smaller $tij$ will receive help from robot i, based on Eq. (1), and it will update Ω_n, Ω_h, and Ω_u. If neither neighbor sends a positive response initially, robot i will offer to help a second time (n_req = 2). If it has not won a bid in the second iteration, it will enter the next decision layer. The bid offering process is shown in lines 5–16 of Table 2.

In Algorithm 3 (Table 3), robot i has finished harvesting a section of length l_i and checks to see if a neighboring robot in row(j) ∈ [row(i)⁺, row(i)⁻] has offered to help (line 9 of Algorithm 1). Since the time incentives sent by the robot(s) on the far side of a row will be larger than that of the one(s) just on the near side and its motion is constrained by the field layout, it will decline the help offer immediately, and only focus on selecting between those that are closest to itself. Robot i compares the time incentive $tji$ to the time it would spend harvesting the remainder of the row by itself $tie$ and informs robot j if it won or not. If more than one time incentive was received from robots in different rows and robot j declines to come help after winning a bid, robot i will select the other time incentive $tji$ accordingly. If the second neighboring robot also declines to help, robot i will not be helped. A robot offering help will repeat the offer a second time if it fails to win a bid in the first iteration, therefore it is necessary for robot i to check for a second offer before resuming Algorithm 1. In this algorithm, the time incentive value for robot i to harvest its current row by itself is computed as $tie=Li(1−λi)/vi,h$⁠. It is noting that the harvesting time is included in the time incentive calculated in Algorithm 2 so to compare with robot i itself in Algorithm 3.

Figure 2 is shown with two example scenarios of a robot selecting rows affiliated with the lowest travel time as an unharvested row (left), and a partially harvested row occupied by a neighbor (right).

Algorithm 4 begins when a robot fails to receive an assignment from its neighbors and requests an assignment from the collection truck. The truck’s computer will wait t_w (e.g., between 1 and 3) seconds after receiving the first inquiry to allow other robots to submit requests before sending out the list of rows in Ω_n and Ω_u (line 3 of Table 4). The computer will then wait for t_c seconds for the robots to calculate and send time incentives. The time incentives are used to determine the row assignments via the Munkres Assignment Algorithm [22] in line 5 of Table 4. The collection truck computer will then update sets Ω_n, Ω_h, and Ω_u, shown in line 7. As mentioned before, Algorithm 4 is centralized; however, this algorithm has a low chance of being used. If the candidate row j belongs to Ω_n, the time incentive for robot i to move from its current row end to that row is $tij=γ|j−row(i)|di,j/vi,nh$⁠. Mimicking a human picker (Assumption 1), the discount factor γ > 1 is used to show the hesitation of a robot to go to a remote row. If the candidate row j belongs to Ω_u, the time incentive for robot i to move from its current row end to the remaining middle of row j is $tij=γ|j−row(i)|di,j/vi,nh+Lj(1−λj)/2/vi,nh$⁠.

Here, the tri-layered, decentralized auction-based row allocation algorithm is abbreviated as DARA, and the decision process is shown in Fig. 3.

Information will be exchanged among robots and/or trucks periodically. A robot will not update its global information such as the row sets until it reaches a truck. In the communication complexity analysis (the worst case scenario), we assume that any information regardless of its type (e.g., Booleans, integers, or floating point numbers) is stored as one unit.

In Algorithm 1 (Table 5), robot i will read information from its onboard file, which is updated by the collection trucks and its neighbors. The only information that will be transmitted from robot i to a truck is the robot index (one unit) and a Boolean (one unit) indicating if repairs, a battery replacement, etc., are needed.

In Algorithm 2, robot i sends an array of three units (a time incentive, its corresponding row, and the robot index) to a maximum of two available neighbors. The neighboring robots respond by each sending two units of information (a Boolean and its index). If no bid is won, robot i resends the offers as three units of information and receives two units of information from each neighbor. Robot i then responds to each of its neighbors with two units of information and informs the truck of the row status changes by sending three units of information (the row that has been completed, the row it is moving to, and its index).

In Algorithm 3, robot i evaluates up to four time incentives sent by neighbors. It will immediately respond to two neighbors with a Boolean and an index (two units) to remove them from the bidding process and will send an additional two-unit response to the robot with the smallest time incentive. Robot i then receives a Boolean response and index (two units). Two units will be sent to its other neighbor if the initial response declines the offer, prompting it to send a two-unit response.

In Algorithm 4, as shown in Fig. 4, the collection truck receives a robot index and an integer prompting the algorithm to execute (two units) from a maximum of N_hr robots. Then the truck will send N_r integers indicating which rows are unharvested (N_r is the number of rows in Ω_n) and the truck index to each of the N_hr robots. It will then receive the robot index and N_r time incentives corresponding to the unharvested rows from the N_hr robots and will send back two units of information to each robot containing the robot index and an integer representing the new row assignment.

Consider a general instance where n₁ robots, 1 ≤ n₁ ≤ N_hr, finished their rows and requested new assignments at around the same time. According to DARA, a robot requesting a new row will always receive an assignment if $Ωn≠∅$ or $Ωu≠∅$⁠. If $Ωn=∅$ and $Ωu=∅$⁠, the robot will wait in the farm until all robots finished their current rows.

The maximum time for the algorithm to converge is defined as the time needed for all the related robots (n₁ robots, their neighbors, and the trucks) to execute the algorithms including communication time, code execution time, and waiting time. Two cases are considered here. In case 1, a robot is seeking an assignment; in case 2, a robot receives a message from its neighbor offering help. The time needed for a truck to run its algorithm is included in these two cases.

Case 1: The robot enters Algorithm 2 to offer help to its neighbors and will enter Algorithm 4 if it does not win a bid. The Munkres Assignment Algorithm [22] in layer 4 guarantees an assignment for each robot requesting a new row if one is available. Thus, the robot will have a fixed number of instances of communication to obtain an assignment. The waiting time and the code execution time in all three layers are fixed and upper bounded. Therefore, the maximum time for a robot to obtain a new assignment is finite.

Case 2: The robot enters Algorithm 3 overseeing two iterations of the bidding process for up to four neighbors. In each iteration, a robot completes a fixed number of instances of communication for each neighbor. Combining the initial reception of help messages and the instances of rejecting robots on the far sides, there are also a fixed total instances of communication. The process finishes after the second iteration regardless of the results of the bidding process. Therefore, a robot receiving a help offer will oversee the bidding process in finite time.

Combining these two cases, the algorithm will converge in finite time.

The computational complexities of the two cases mentioned in Sec. 4.2 are analyzed here.

Case 1: The CPU time used in calculating the time incentives is constant since only addition and multiplication operations are used. Communicating information about new rows is of constant complexity. Calculating time incentives corresponding to these rows is of O(n₂) complexity, where n₂ is the number of rows related to n₁ robots and is bounded by the size of the strawberry field. The complexity of the centralized process, the Munkres Assignment Algorithm, has a $O(n33)$ time complexity [22], where n₃ is the larger of the number of rows in Ω_n and Ω_u and the number of robots requesting a new assignment. Therefore, the upper bound of time complexity for case 1 is $O(n33)$⁠.

Case 2: The robot receives a help inquiry and oversees the bidding process, which is upper bound by the complexity of Algorithm 3 process. Reading in the time incentives, comparing these values, and communicating the results, each have constant time complexities. The process is limited to two iterations, resulting in an overall constant time complexity.

Combining both cases, the complexity of the centralized part in DARA is $O(n33)$⁠, while that of the decentralized part is O(n₁). The chances of using the centralized part are low.

The Monte Carlo simulation aims to evaluate the ability of DARA to produce a new row assignment within the constraints. Table 6 lists the constant settings related to a simulated farm and harvesting robot fleet.

Different scheduling scenarios are mimicked by different combinations of the independent variables shown in Table 7. For example, a “repair” event is triggered when the Boolean representing a mechanical failure, h, is set to 1 or the battery life η (between 0% and 100%) is below a threshold. The determination of h is biased in favor of 0, as robots are not expected to fail frequently. Variables such as ζ, λ, and η are assigned with a normal distribution centered at 50%. When the variable indicating the percent fullness of a robot’s basket, ζ, is at 100%, it will trigger the robot to deliver the basket to a nearby collection truck. The number of robots in the fleet is proportionate to the size of the farm ranging between 30 and 480 rows.

Each simulated robot is assigned to a randomly selected row; however, no more than two robots will be assigned to the same row. To reliably trigger a row assignment event, we let robot i always be at the end of its row, with other robots in the fleet given a random overbed position λ ranging between 0% and 100% of the row length. Since robots are likely to progress through the field near the rest of the fleet, robot i in row(i) has two available neighbors, with other robots being randomly assigned to nearby rows.

Ω_u, Ω_n, Ω_h, and time incentives are the dependent variables, which are computed based on the independent variables in Table 7. Set Ω_u is determined by row(i) in Table 7 and set Ω_n includes rows randomly selected from all the rows excluding those in set Ω_u.

The Monte Carlo simulation of DARA is executed for 100,000 trials and compared to a benchmark central process (Benchmark). Benchmark utilizes only the centralized Munkres Assignment Algorithm [22] to award assignments from a set of available rows and is evaluated under the same conditions as DARA. The actual row assignment for each simulation run is not shown which is not crucial in demonstrating the advantages of the algorithm against the benchmark algorithm.

In Table 8, Benchmark and DARA assignments and execution times are observed for varying farm and fleet sizes. DARA assignments are globally optimal when they match the Benchmark assignments, which occurs a majority of the time. Discrepancies are caused by DARA assigning rows within the scope of its neighbors and Benchmark assigning unharvested rows outside of this range (e.g., the number of robots in the fleet) in cases where the overbed distance to travel to the starting point of an partially harvested row is significant. In these cases, DARA assignments are locally optimal. The results of the simulation indicate that, at minimum, the locally optimal assignment is obtained using DARA when compared to Benchmark with 100% of trials resulting in successful row assignments. Figure 5 shows the average execution times from Table 8 as the numbers of rows and robots increase, with DARA outperforming Benchmark and displaying much better scalability. It is worth noting that the waiting times are not included in the execution times.

The applicability of algorithm is mainly constrained by the assumptions listed in Sec. 2. As one example, if Assumption 1 is invalid, meaning the total number of strawberries harvested is not proportional to the length a robot covered, the result achieved via this algorithm may only be a feasible solution. In this case, a stochastic optimization algorithm will need to be considered, which is beyond the scope of this study. Additionally, if collection trucks are only present on one side of the field (against Assumption 8), the distance calculation will be different. In this case, the algorithm is still applicable, but the results will be different.

The row assignment problem for a fleet of cooperative, small harvesting robots in strawberry fields is studied including local negotiations and a centralized fallback algorithm. The algorithm allocates a new row to a robot based on the time the robot predicts it will take to complete the row. The algorithm is validated in a Monte Carlo simulation and compared to a benchmark centralized algorithm. The results show that the algorithm finds local optimal solutions and has a lesser computational cost with superior scalability.

This work was supported by the National Science Foundation (Grant No. 1924622).

There are no conflicts of interest.