Paginate order fetches in Live Reconciliation tutorial (#2300)

04 Research Environment/10 Meta Analysis/05 Live Reconciliation/05 Plot Order Fills.php

@@ -1,37 +1,67 @@
 <p>Follow these steps to overlay live and OOS backtest order fills on a single marker-only chart per symbol. The chart deliberately omits candlesticks and any price history so the comparison between live and backtest executions is not drowned out by other series.</p>
 
 <ol>
-    <li>Read the live and backtest orders. Both endpoints can take a few seconds to load the first time, so retry until the response is non-empty.</li>
+    <li>Read the live and backtest orders. Each call to <code class="csharp">ReadLiveOrders</code><code class="python">read_live_orders</code> and <code class="csharp">ReadBacktestOrders</code><code class="python">read_backtest_orders</code> returns at most 100 orders, so paginate in 100-Id windows until the endpoint returns an empty window. The first window can take a few seconds to load, so retry while it is empty.</li>
     <div class="section-example-container">
-        <pre class="csharp">List&lt;ApiOrderResponse&gt; ReadOrders(Func&lt;List&lt;ApiOrderResponse&gt;&gt; fetch)
+        <pre class="csharp">List&lt;ApiOrderResponse&gt; ReadAllOrders(Func&lt;int, int, List&lt;ApiOrderResponse&gt;&gt; fetchWindow)
 {
+    var all = new List&lt;ApiOrderResponse&gt;();
+    // Retry the first window while the response is empty (may be loading).
+    List&lt;ApiOrderResponse&gt; first = null;
     for (var attempt = 0; attempt &lt; 10; attempt++)
     {
-        var result = fetch();
-        if (result.Any()) return result;
+        first = fetchWindow(0, 100);
+        if (first.Any()) break;
         Console.WriteLine($"Orders loading... (attempt {attempt + 1}/10)");
         Thread.Sleep(10000);
     }
-    throw new Exception("Failed to read orders after 10 attempts");
+    if (first == null || !first.Any()) return all;
+    all.AddRange(first);
+    // Paginate in 100-Id windows until the endpoint returns an empty window.
+    var start = 100;
+    while (true)
+    {
+        var window = fetchWindow(start, start + 100);
+        if (!window.Any()) break;
+        all.AddRange(window);
+        start += 100;
+    }
+    return all;
 }
 
-var liveOrders = ReadOrders(() =&gt; api.ReadLiveOrders(projectId, 0, 100));
-var backtestOrders = ReadOrders(() =&gt; api.ReadBacktestOrders(projectId, backtestId, 0, 100));</pre>
+var liveOrders = ReadAllOrders((s, e) =&gt; api.ReadLiveOrders(projectId, s, e));
+var backtestOrders = ReadAllOrders((s, e) =&gt; api.ReadBacktestOrders(projectId, backtestId, s, e));
+Console.WriteLine($"Live orders: {liveOrders.Count}, OOS orders: {backtestOrders.Count}");</pre>
         <pre class="python">from time import sleep
 
-def read_orders(fetch):
+def read_all_orders(fetch_window):
+    orders = []
+    # Retry the first window while the response is empty (may be loading).
+    first = []
     for attempt in range(10):
-        result = fetch()
-        if result:
-            return result
+        first = fetch_window(0, 100)
+        if first:
+            break
         print(f"Orders loading... (attempt {attempt + 1}/10)")
         sleep(10)
-    raise RuntimeError("Failed to read orders after 10 attempts")
-
-live_orders = read_orders(lambda: api.read_live_orders(project_id, 0, 100))
-backtest_orders = read_orders(lambda: api.read_backtest_orders(project_id, backtest_id, 0, 100))</pre>
+    if not first:
+        return orders
+    orders.extend(first)
+    # Paginate in 100-Id windows until the endpoint returns an empty window.
+    start = 100
+    while True:
+        window = fetch_window(start, start + 100)
+        if not window:
+            break
+        orders.extend(window)
+        start += 100
+    return orders
+
+live_orders = read_all_orders(lambda s, e: api.read_live_orders(project_id, s, e))
+backtest_orders = read_all_orders(lambda s, e: api.read_backtest_orders(project_id, backtest_id, s, e))
+print(f"Live orders: {len(live_orders)}, OOS orders: {len(backtest_orders)}")</pre>
     </div>
-    <p>By default, you receive the orders with an Id between 0 and 100. To read more, call the method repeatedly in windows of up to 100 Ids. For more on the order objects returned, see <a href='/docs/v2/research-environment/meta-analysis/live-analysis#03-Plot-Order-Fills'>Plot Order Fills</a> in the Live Analysis documentation.</p>
+    <p>For more on the order objects returned, see <a href='/docs/v2/research-environment/meta-analysis/live-analysis#03-Plot-Order-Fills'>Plot Order Fills</a> in the Live Analysis documentation.</p>
 
     <li>Organize the trade times and prices for each security into a dictionary for both the live and backtest fills.</li>
     <div class="section-example-container">

04 Research Environment/10 Meta Analysis/05 Live Reconciliation/99 Examples.html

@@ -59,21 +59,34 @@ <h4>Example 1: Generate an OOS Reconciliation Curve</h4>
 // Read the backtest Strategy Equity chart.
 var backtestEquity = api.ReadBacktestChart(projectId, "Strategy Equity", 0, nowSec, 500, backtestId).Chart;
 
-// Read live and backtest orders for the fill overlay. Both endpoints can take
-// a few seconds to load the first time, so retry until the list is non-empty.
-List&lt;ApiOrderResponse&gt; ReadOrders(Func&lt;List&lt;ApiOrderResponse&gt;&gt; fetch)
+// Read live and backtest orders for the fill overlay. Each call returns at
+// most 100 orders, so paginate in 100-Id windows until we get an empty window.
+// The first window can take a few seconds to load, so retry while empty.
+List&lt;ApiOrderResponse&gt; ReadAllOrders(Func&lt;int, int, List&lt;ApiOrderResponse&gt;&gt; fetchWindow)
 {
+    var all = new List&lt;ApiOrderResponse&gt;();
+    List&lt;ApiOrderResponse&gt; first = null;
     for (var attempt = 0; attempt &lt; 10; attempt++)
     {
-        var result = fetch();
-        if (result.Any()) return result;
+        first = fetchWindow(0, 100);
+        if (first.Any()) break;
         Thread.Sleep(10000);
     }
-    throw new Exception("Failed to read orders after 10 attempts");
+    if (first == null || !first.Any()) return all;
+    all.AddRange(first);
+    var start = 100;
+    while (true)
+    {
+        var window = fetchWindow(start, start + 100);
+        if (!window.Any()) break;
+        all.AddRange(window);
+        start += 100;
+    }
+    return all;
 }
 
-var liveOrders = ReadOrders(() =&gt; api.ReadLiveOrders(projectId, 0, 100));
-var backtestOrders = ReadOrders(() =&gt; api.ReadBacktestOrders(projectId, backtestId, 0, 100));
+var liveOrders = ReadAllOrders((s, e) =&gt; api.ReadLiveOrders(projectId, s, e));
+var backtestOrders = ReadAllOrders((s, e) =&gt; api.ReadBacktestOrders(projectId, backtestId, s, e));
 
 Console.WriteLine($"Start: {startDate}, Starting cash: {startingCash}, End: {endDate}");
 Console.WriteLine($"Live orders: {liveOrders.Count}, OOS orders: {backtestOrders.Count}");
@@ -235,16 +248,28 @@ <h4>Example 1: Generate an OOS Reconciliation Curve</h4>
         (d.buy_fill_prices if is_buy else d.sell_fill_prices).append(order.price)
     return out
 
-def read_orders(fetch):
+def read_all_orders(fetch_window):
+    orders = []
+    first = []
     for attempt in range(10):
-        result = fetch()
-        if result:
-            return result
+        first = fetch_window(0, 100)
+        if first:
+            break
         sleep(10)
-    raise RuntimeError("Failed to read orders after 10 attempts")
+    if not first:
+        return orders
+    orders.extend(first)
+    start = 100
+    while True:
+        window = fetch_window(start, start + 100)
+        if not window:
+            break
+        orders.extend(window)
+        start += 100
+    return orders
 
-live_by_symbol = group_by_symbol(read_orders(lambda: api.read_live_orders(project_id, 0, 100)))
-backtest_by_symbol = group_by_symbol(read_orders(lambda: api.read_backtest_orders(project_id, backtest_id, 0, 100)))
+live_by_symbol = group_by_symbol(read_all_orders(lambda s, e: api.read_live_orders(project_id, s, e)))
+backtest_by_symbol = group_by_symbol(read_all_orders(lambda s, e: api.read_backtest_orders(project_id, backtest_id, s, e)))
 
 for symbol in set(live_by_symbol) | set(backtest_by_symbol):
     live = live_by_symbol.get(symbol, OrderData())

09 AI Assistance/03 MCP Server/01 Key Concepts/04 Tools.html

@@ -49,3 +49,39 @@ <h4>Live Trading</h4>
   <li><code>stop_live_algorithm</code> - Stop a live algorithm.</li>
   <li><code>liquidate_live_algorithm</code> - Liquidate and stop a live algorithm.</li>
 </ul>
+
+<h4>Research</h4>
+<p>The following tools operate on a Jupyter notebook in the QuantConnect Cloud Research environment:</p>
+<ul>
+  <li><code>jupyter_create_cell</code> - Insert a new Jupyter notebook cell. By default the cell is appended; provide an index to insert at a specific zero-based position.</li>
+  <li><code>jupyter_read_cell</code> - Return the source lines for a single notebook cell at a zero-based index.</li>
+  <li><code>jupyter_update_cell</code> - Replace the source of an existing notebook cell at a zero-based index.</li>
+  <li><code>jupyter_delete_cell</code> - Delete a notebook cell at a zero-based index.</li>
+  <li><code>jupyter_execute_cell</code> - Execute one code cell by zero-based index and return its captured outputs.</li>
+  <li><code>jupyter_create_notebook</code> - Replace the current notebook with raw notebook JSON content.</li>
+  <li><code>jupyter_read_notebook</code> - Return the full current notebook as a raw JSON string.</li>
+  <li><code>jupyter_execute_notebook</code> - Execute all notebook cells in order and return aggregated outputs from code cells.</li>
+</ul>
+
+<h4>News</h4>
+<p>The following tools fetch financial news and blog posts from a curated list of more than 20 sources:</p>
+<ul>
+  <li><code>financial_data_blog_posts</code> - Fetch recent financial blog posts.</li>
+  <li><code>financial_data_web_get</code> - Fetch the full content of a web page given its URL. Use this to read blog posts and articles.</li>
+</ul>
+
+<h4>Environment</h4>
+<ul>
+  <li><code>environment_library_support</code> - Fetch the list of available preinstalled libraries in the QuantConnect environment.</li>
+  <li><code>datasets</code> - Fetch the list of available QuantConnect datasets.</li>
+</ul>
+
+<h4>Notifications</h4>
+<p>
+  The following tools send messages from QuantConnect Cloud to you using the <a href='/docs/v2/writing-algorithms/live-trading/notifications'>Live Trading Notifications</a> system:
+</p>
+<ul>
+  <li><code>send_email_notification</code> - Send an email notification.</li>
+  <li><code>send_sms_notification</code> - Send an SMS notification.</li>
+  <li><code>send_telegram_notification</code> - Send a Telegram notification.</li>
+</ul>