Certified Cloud Developer


Seit meinem Einstieg in die Softwareentwicklung in den 80er-Jahren habe ich mich kontinuierlich in der jeweils aktuellen Technologie bewegt: von Assembler über C++ und Lisp/Common Lisp in den 1980er-Jahren, gefolgt von Java und Python in den 1990er-Jahren, später C# und .NET. Seit 2024 widme ich mich verstärkt wieder der Softwareentwicklung, nun im Kontext von KI-gestützten Systemen.

Als Anthropic-Certified Claude Developer setze ich moderne KI-Entwicklungsmethoden produktiv ein: Aufgaben, die früher Wochen an Entwicklungszeit beansprucht hätten, lassen sich heute in Stunden umsetzen. Aktuelles Beispiel ist Kontor, eine von mir entwickelte Handelsoberfläche mit Claude-API-Anbindung und einem eigenständigen Trader-Agent für KI-gestützte Marktanalyse, inklusive eines öffentlichen Wirtschaftskalenders mit allen Terminen.

Engineering is a Passion


Als System-Engineer (MCITP, CCP-V und AI Fluency & Claude Expert) betreue ich Netzwerkinfrastrukturen, führe Planungen durch, schule Mitarbeiter und führe Migrationen von Netzwerk, Server- als auch Client Infrastrukturen bis zum mittleren Enterprise Umfeld durch. Mein Aufgabengebiet umfasste der IT Support von mittelständischen Unternehmen als auch die Umsetzung von Projekten in Server und Clientnetzwerken unter den Betriebssystemen Microsoft, Mac OS und Linux resp. den Hypervisoren VMware, Hyper V und Citrix.

Mein Knowhow und Wissen sammelte ich in Projekten im Bereich öffentlicher und privater Auftraggeber.

 

Backup- und Restore-Skripte

← Zurück zum Artikel: Microsoft365DSC, Teil 2: Inhalte und Secrets sichern

Die Skripte aus dem Artikel zum Lesen, Ändern und Mitnehmen. Änderungen bleiben in diesem Browserfenster. Es wird nichts gespeichert oder gesendet.

Zum Bearbeiten am Rechner öffnen.

Lesefassung ohne Editor
README.mdMarkdown · 7,0 kB
# Microsoft-365-Backup (nur lesend) mit Restore der Secrets

Skripte, die die Konfiguration und die Inhalte eines Microsoft-365-Tenants lesend als JSON und Dateien sichern:
Verzeichnis (Benutzer, Gruppen, Rollen, Apps), Richtlinien, Intune, SharePoint (Sites, Listen, Dateien, Versionen),
Postfächer, Teams, Exchange-Online- und Compliance-Konfiguration. Alle Aufrufe gegen Microsoft sind lesend (GET bzw. `Get-*`).
Dazu kommt ein Restore-Skript, das die **nicht sicherbaren** Kennwörter und Secret-Werte beim Wiederherstellen neu setzt.

## Voraussetzungen

- Python 3 mit `requests`, `PyJWT` und `cryptography`. Für die Zertifikate im Restore zusätzlich `openssl`.
- PowerShell 7 mit den Modulen `ExchangeOnlineManagement` und `MicrosoftTeams` (nur für `compliance.ps1` und `teams_settings.ps1`).
- Eine **App-Registrierung** im Tenant (Beispielname `backup-reader`) mit Anwendungsberechtigungen, die nur lesen
  (zum Beispiel `User.Read.All`, `Group.Read.All`, `Directory.Read.All`, `Policy.Read.All`, `Application.Read.All`, `Sites.Read.All`,
  `Files.Read.All`, `Mail.Read`, `Calendars.Read`, `Contacts.Read`, `Team.ReadBasic.All`, `Chat.Read.All`, `ChannelMessage.Read.All`,
  `DeviceManagementConfiguration.Read.All`, `DeviceManagementApps.Read.All`, `DeviceManagementManagedDevices.Read.All`),
  dazu für Exchange `Exchange.ManageAsApp` mit einer Leserolle und für SharePoint `Sites.FullControl.All` nur dann, wenn die Admin-Site gelesen werden soll.
  Welche Rechte wirklich ankommen, zeigt `backup/permission_probe.py`.
- Ein **Zertifikat** für die App: öffentlicher Teil in der App-Registrierung, privater Schlüssel (PEM) auf dem Backup-Rechner.

## Umgebungsvariablen

Es gibt keine festen Werte im Code. Alles kommt aus der Umgebung:

| Variable | Inhalt |
|---|---|
| `M365_TENANT_ID` | Verzeichnis-ID, `<TENANT-ID>` |
| `M365_CLIENT_ID` | Anwendungs-ID der App, `<CLIENT-ID>` |
| `M365_CERT_THUMBPRINT` | SHA-1-Fingerabdruck des Zertifikats, `<CERT-THUMBPRINT>` |
| `M365_CERT_KEY_PATH` | Pfad zum privaten Schlüssel (PEM); ersatzweise `M365_CERT_PATH` |
| `M365_CERT_PATH` | Pfad zum Zertifikat (PEM), nur für die PowerShell-Skripte |
| `M365_TENANT_NAME` | Tenant-Domäne, zum Beispiel `contoso.onmicrosoft.com` |
| `M365_ANCHOR_GUID` | optional: GUID der System-Mailbox für den Header `X-AnchorMailbox` |

Für den Restore gelten eigene Variablen für den **Ziel**-Tenant: `M365_TARGET_CLIENT_SECRET` oder
`M365_TARGET_CERT_PEM` und `M365_TARGET_CERT_KEY`. Fehlt beides, fragt das Skript das Client-Secret mit `getpass` ab.

## Reihenfolge

Alle Skripte nehmen `--out <Zielordner>` (zum Beispiel `./backup/contoso/20261116-0557`) und lassen sich mit `--help` ansehen.
Die meisten sind wiederaufnehmbar: Bereits gesicherte Dateien werden übersprungen. Mit `--max-*` lässt sich der Umfang für einen Probelauf begrenzen.

1. `backup/graph_export.py` Verzeichnis, Richtlinien, Rollen, Apps, Intune, SharePoint-Sites mit Listen (nach `graph/`).
2. `backup/drive_inventory.py` Inventar aller Laufwerke (nach `graph/drives_inventory.json`).
3. `backup/files_export.py` alle Dateien samt `manifest.jsonl` (SHA-256).
4. `backup/file_versions.py` ältere Dateiversionen (`--max-versions`).
5. `backup/sharepoint_lists.py` Listeneintrag-Versionen, Papierkorb, Seiten mit Layout, Strukturprüfung.
6. `backup/mailboxes.py` Mail, Anhänge, Kalender und Kontakte je Benutzer.
7. `backup/teams.py` Teams, Kanäle, Nachrichten, Chats.
8. `backup/graph_settings.py` weitere Richtlinien, Rollen, Intune-Details, Lizenzen.
9. `backup/spo_tenant.py` SharePoint-Tenant-Einstellungen über die Admin-Site.
10. `backup/exchange.py` Exchange-Online-Konfiguration (`--retry-gzip` wiederholt Abrufe, die an einem gzip-Fehler scheiterten).
11. `backup/compliance.ps1` Purview/Compliance-Konfiguration (PowerShell).
12. `backup/teams_settings.ps1` Teams-Richtlinien (PowerShell).
13. `backup/secrets_inventory.py` Inventar der App-Secrets und Zertifikate (nur Metadaten).
14. `backup/permission_probe.py`, `backup/verify_files.py`, `backup/report.py` Rechteprobe, Vollständigkeitsprüfung, Sammelbericht.

`m365_auth.py` ist die gemeinsame Anmeldung (Zertifikat-JWT, Wiederholung bei 429/5xx, Paging). Exchange-Aufrufe sind auf `Get-*` beschränkt.

## Was gesichert wird und was nicht

Gesichert werden Konfiguration und Inhalte, soweit die Berechtigungen reichen. Was Microsoft nicht herausgibt, fehlt:
**Kennwörter, Secret-Werte von Apps und private Schlüssel von Zertifikaten sind nicht auslesbar.** Das Inventar
(`secrets_inventory.json`) hält nur fest, welche Secrets und Zertifikate es gab: App, Name, `keyId`, Ablaufdatum.
Ebenfalls nicht enthalten sind MFA-Registrierungen, Verbindungen (Power Automate und andere), Gast-Einladungen und
Inhalte, die der Tenant per API nicht liefert. Was ein Endpunkt verweigert hat, steht in den Dateien `not_possible.json`.

## Restore der Secrets

`restore/secrets_restore.py` liest den Plan aus dem Backup und zeigt, welche Stellen ein neues Secret brauchen
(Benutzerkennwörter, App-Secrets, Zertifikate, Dienstprinzipale, dazu eine Checkliste für manuelle Schritte).

- **Ohne `--execute` ist alles ein Trockenlauf:** kein Netz, keine Änderung, nur der Plan.
- Mit `--execute` fragt das Skript **interaktiv** nach: je Person «random», «enter» oder «skip». Eigene Kennwörter werden
  zweimal mit `getpass` eingegeben (Eingabe unsichtbar) und gegen Regeln geprüft (12 bis 256 Zeichen, 3 von 4 Zeichenarten).
- Neue App-Secrets und Zertifikate werden im Ziel angelegt. Die neuen Werte stehen **nie** in Journal oder Log. Sie landen
  in einer Datei mit Rechten 600 (oder einmalig am Bildschirm, optional zusätzlich im Azure Key Vault) und müssen danach
  in einen Kennwort-Manager übernommen und die Datei gelöscht werden.
- Wiederaufnahme: Das Journal merkt sich Erledigtes. Ein erneuter Start überspringt es.

Beispiel Trockenlauf: `python3 restore/secrets_restore.py --backup ./backup/contoso/20261116-0557 --tenant <TENANT-ID> --client-id <CLIENT-ID> --new-domain example.com`

## Prüfstand (ehrlich)

Der Demo-Tenant ist abgelaufen. Seit der Umbenennung und Übersetzung ist **kein Skript gegen einen Tenant gelaufen**.
Geprüft wurde:

- Syntax aller Python-Dateien (`py_compile`) und `--help` jedes Skripts ohne Anmeldung.
- Die eingebauten Unittests des Restore (7 Tests, `python3 -m unittest secrets_restore` im Ordner `restore/`).
- Ein Trockenlauf des Restore gegen ein vorhandenes Backup, nur lesend und ohne Netz.

Nicht geprüft: alle Aufrufe gegen Microsoft (Graph, Exchange-Admin-REST, SharePoint-Admin), die zwei PowerShell-Skripte
(`compliance.ps1`, `teams_settings.ps1`; PowerShell stand nicht zur Verfügung), das Schreiben ins Ziel beim Restore
und die Ablage im Key Vault. Neu gegenüber der früheren Fassung ist der eigene Zugriff auf die SharePoint-Admin-Site in
`spo_tenant.py` (vorher ein lokales Hilfsmodul, das hier nicht dabei ist). Die Sicherungsdateien haben jetzt englische
Schlüssel und Ordnernamen; Skripte, die Daten eines früheren Laufs lesen, brauchen einen Lauf mit der neuen Fassung.

Die Skripte sind ein Arbeitsstand ohne Gewähr; Einsatz und Folgen liegen bei Ihnen.
backup/compliance.ps1PowerShell · 4,0 kB
<#
.SYNOPSIS
Sichert die Compliance-Konfiguration (Purview) über Security & Compliance PowerShell, nur mit Get-Cmdlets.

.DESCRIPTION
Meldet sich mit der App-Registrierung und dem Zertifikat an (Connect-IPPSSession) und schreibt je Cmdlet
eine JSON-Datei in den Zielordner, dazu einen Bericht (_compliance_report.json).
Danach werden die eigenen Informationstypen (Publisher ungleich Microsoft) und die Audit-Konfiguration gesichert.
Es werden keine Einstellungen geändert.

Umgebungsvariablen: M365_CLIENT_ID, M365_TENANT_NAME (z. B. contoso.onmicrosoft.com),
M365_CERT_PATH (Zertifikat, PEM) und M365_CERT_KEY_PATH (privater Schlüssel, PEM).

.PARAMETER OutDir
Zielordner, normalerweise <backup>/compliance.

.EXAMPLE
./compliance.ps1 -OutDir ./backup/contoso/20261116-0557/compliance
#>
param([Parameter(Mandatory)][string]$OutDir)

foreach ($name in 'M365_CLIENT_ID', 'M365_TENANT_NAME', 'M365_CERT_PATH', 'M365_CERT_KEY_PATH') {
    if (-not (Get-Item "env:$name" -ErrorAction SilentlyContinue)) { throw "environment variable $name is not set" }
}
New-Item -ItemType Directory -Force -Path $OutDir | Out-Null
Install-Module ExchangeOnlineManagement -Scope CurrentUser -Force 2>&1 | Out-Null
$cert = [System.Security.Cryptography.X509Certificates.X509Certificate2]::CreateFromPemFile($env:M365_CERT_PATH, $env:M365_CERT_KEY_PATH)
Connect-IPPSSession -AppId $env:M365_CLIENT_ID -Certificate $cert -Organization $env:M365_TENANT_NAME -ShowBanner:$false

$report = @{}
$cmdlets = 'Get-RetentionCompliancePolicy', 'Get-RetentionComplianceRule', 'Get-DlpCompliancePolicy', 'Get-DlpComplianceRule', 'Get-Label', 'Get-LabelPolicy',
    'Get-ComplianceTag', 'Get-DlpSensitiveInformationType', 'Get-ComplianceCase', 'Get-AuditConfig', 'Get-UnifiedAuditLogRetentionPolicy', 'Get-ProtectionAlert',
    'Get-InformationBarrierPolicy', 'Get-SupervisoryReviewPolicyV2', 'Get-SupervisoryReviewRule', 'Get-RoleGroup', 'Get-ComplianceSecurityFilter',
    'Get-DlpKeywordDictionary', 'Get-AutoSensitivityLabelPolicy', 'Get-AutoSensitivityLabelRule', 'Get-ComplianceRetentionEventType',
    'Get-OrganizationSegment', 'Get-PolicyConfig', 'Get-ComplianceSearch', 'Get-ManagementRoleAssignment', 'Get-AdminAuditLogConfig',
    'Get-DlpSensitiveInformationTypeRulePackage', 'Get-FilePlanPropertyAuthority', 'Get-FilePlanPropertyCitation', 'Get-FilePlanPropertyDepartment',
    'Get-FilePlanPropertyReferenceId', 'Get-FilePlanPropertyCategory', 'Get-FilePlanPropertySubCategory', 'Get-ComplianceRetentionEvent',
    'Get-RecordReviewNotificationTemplateConfig', 'Get-IRMConfiguration'
foreach ($c in ($cmdlets | Select-Object -Unique)) {
    try {
        $p = @{ ErrorAction = 'Stop' }
        if ($c -eq 'Get-Label') { $p.IncludeDetailedLabelActions = $true }
        $r = @(& $c @p)
        $r | ConvertTo-Json -Depth 8 -WarningAction SilentlyContinue | Set-Content "$OutDir/$($c.Substring(4)).json"
        $report[$c] = @{ count = $r.Count }
        "OK $c $($r.Count)"
    }
    catch {
        $m = $_.Exception.Message
        if ($m.Length -gt 300) { $m = $m.Substring(0, 300) }
        $report[$c] = @{ error = $m }
        "ERROR $c $m"
    }
}
$report | ConvertTo-Json -Depth 4 | Set-Content "$OutDir/_compliance_report.json"

# Zweiter Teil: eigene Informationstypen und Audit-Konfiguration
$all = @(Get-DlpSensitiveInformationType)
"all: $($all.Count)"
$custom = @($all | Where-Object { $_.Publisher -ne 'Microsoft Corporation' })
"custom: $($custom.Count)"
$custom | Select-Object Name, Id, Publisher, Type, Description, RulePackId, Locale, RecommendedConfidence |
    ConvertTo-Json -Depth 4 | Set-Content "$OutDir/DlpSensitiveInformationType_custom.json"
$all | Select-Object Name, Id, Publisher, Type | ConvertTo-Json | Set-Content "$OutDir/DlpSensitiveInformationType_names.json"
foreach ($c in 'Get-AuditConfigurationPolicy', 'Get-AuditConfig') {
    try {
        $r = @(& $c -ErrorAction Stop)
        "OK $c $($r.Count)"
        $r | ConvertTo-Json -Depth 5 | Set-Content "$OutDir/$($c.Substring(4)).json"
    }
    catch {
        "ERROR $c " + $_.Exception.Message.Substring(0, [Math]::Min(200, $_.Exception.Message.Length))
    }
}
backup/drive_inventory.pyPython · 1,5 kB
"""Inventar aller Laufwerke (Drives) der gesicherten SharePoint-Sites per Delta-Abfrage.
Liest graph/sharepoint/*.json aus dem Zielordner und schreibt graph/drives_inventory.json.
Nur GET."""
import glob, json, os, sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth

SELECT = "id,name,size,parentReference,file,folder,webUrl,lastModifiedDateTime,createdBy,lastModifiedBy"


def main(argv=None):
    p = auth.make_parser("Inventory of all drives of the exported SharePoint sites")
    args = p.parse_args(argv)
    out = {}
    # Der graph_export schreibt die Sites nach <out>/graph/sharepoint/
    for f in glob.glob(os.path.join(args.out, "graph", "sharepoint", "*.json")):
        j = json.load(open(f))
        for dr in (j["drives"] if isinstance(j["drives"], list) else []):
            items, url = [], f"/drives/{dr['id']}/root/delta?$select={SELECT}"
            while url:
                r = auth.g("GET", url)
                if r.status_code != 200:
                    items = {"error": r.status_code}; break
                jj = r.json(); items += jj.get("value", []); url = jj.get("@odata.nextLink")
            out[dr["id"]] = {"name": dr.get("name"), "webUrl": dr.get("webUrl"), "items": items}
    os.makedirs(os.path.join(args.out, "graph"), exist_ok=True)
    auth.write_json(os.path.join(args.out, "graph", "drives_inventory.json"), out)
    print(len(out), sum(len(v["items"]) for v in out.values() if isinstance(v["items"], list)))
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/exchange.pyPython · 8,4 kB
"""Exchange Online: Konfiguration und Postfach-Einstellungen über Get-Cmdlets (nur lesend).

Die Cmdlets laufen über die Admin-REST-Schnittstelle (m365_auth.invoke, nur Get-*). Ausgabe nach exchange/<Name>.json,
Bericht nach exchange_report.json. Mit --retry-gzip werden Cmdlets nachgeholt, die im Bericht wegen
eines gzip-Fehlers fehlgeschlagen sind (früher ein eigenes Nachlauf-Skript)."""
import datetime, json, os, sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth

SIMPLE = [
    "Get-OrganizationConfig", "Get-AcceptedDomain", "Get-RemoteDomain", "Get-TransportConfig", "Get-TransportRule",
    "Get-InboundConnector", "Get-OutboundConnector", "Get-HostedContentFilterPolicy", "Get-HostedContentFilterRule",
    "Get-HostedConnectionFilterPolicy", "Get-HostedOutboundSpamFilterPolicy", "Get-HostedOutboundSpamFilterRule",
    "Get-MalwareFilterPolicy", "Get-MalwareFilterRule", "Get-AntiPhishPolicy", "Get-AntiPhishRule", "Get-SafeLinksPolicy", "Get-SafeLinksRule",
    "Get-SafeAttachmentPolicy", "Get-SafeAttachmentRule", "Get-AtpPolicyForO365", "Get-QuarantinePolicy", "Get-GlobalQuarantinePolicy",
    "Get-DkimSigningConfig", "Get-MailboxPlan", "Get-CASMailboxPlan", "Get-DynamicDistributionGroup", "Get-UnifiedGroup", "Get-MailContact", "Get-MailUser",
    "Get-RetentionPolicy", "Get-RetentionPolicyTag", "Get-SharingPolicy", "Get-OwaMailboxPolicy", "Get-MobileDeviceMailboxPolicy",
    "Get-ActiveSyncDeviceAccessRule", "Get-ActiveSyncOrganizationSettings", "Get-RoleGroup", "Get-ManagementRoleAssignment", "Get-ManagementRole",
    "Get-AdminAuditLogConfig", "Get-AuthenticationPolicy", "Get-AddressList", "Get-GlobalAddressList", "Get-OfflineAddressBook",
    "Get-AddressBookPolicy", "Get-JournalRule", "Get-EmailAddressPolicy", "Get-EOPProtectionPolicyRule", "Get-ATPProtectionPolicyRule",
    "Get-ATPBuiltInProtectionRule", "Get-ExoPhishSimOverrideRule", "Get-PhishSimOverridePolicy", "Get-ApplicationAccessPolicy", "Get-AuthServer",
    "Get-IntraOrganizationConnector", "Get-OrganizationRelationship", "Get-MessageClassification", "Get-OutlookProtectionRule",
    "Get-DataClassification", "Get-ReportSubmissionPolicy", "Get-ReportSubmissionRule", "Get-TeamsProtectionPolicy", "Get-TeamsProtectionPolicyRule",
    "Get-MailboxAuditBypassAssociation", "Get-PartnerApplication", "Get-PolicyTipConfig", "Get-RoleAssignmentPolicy", "Get-ResourceConfig",
    "Get-ClientAccessRule", "Get-HybridConfiguration", "Get-AvailabilityConfig", "Get-OnPremisesOrganization", "Get-DeliveryAgentConnector",
    "Get-MalwareFilteringServer", "Get-ServicePrincipal", "Get-ManagementScope", "Get-MigrationEndpoint", "Get-OMEConfiguration", "Get-IRMConfiguration"]
# Hinweis: Diese Cmdlets stehen bewusst nicht in der Liste (liefern nichts, brauchen Parameter oder laufen unten einzeln):
# Get-MailboxFolderPermission, Get-MobileDevice, Get-SendConnector, Get-Recipient, Get-FeatureConfiguration, Get-Place, Get-SmimeConfig.


def main(argv=None):
    p = auth.make_parser("Back up Exchange Online configuration via Get-* cmdlets (read-only)")
    p.add_argument("--retry-gzip", action="store_true", help="only repeat cmdlets whose earlier run failed with a gzip error")
    p.add_argument("--max-mailboxes", type=int, default=0, help="limit per-mailbox cmdlets to N mailboxes (0 = all)")
    args = p.parse_args(argv)
    out = args.out
    os.makedirs(out, exist_ok=True)
    report_path = os.path.join(out, "exchange_report.json")
    report = {"start": datetime.datetime.now(datetime.timezone.utc).isoformat(), "cmdlets": {}, "not_possible": {}}
    log_file = open(os.path.join(out, "exchange.log"), "a")

    def log(*a):
        s = " ".join(str(x) for x in a); log_file.write(s + "\n"); log_file.flush(); print(s, flush=True)

    def save(folder, name, data):
        os.makedirs(os.path.join(out, folder), exist_ok=True)
        with open(os.path.join(out, folder, name + ".json"), "w") as f:
            json.dump(data, f, indent=1, ensure_ascii=False, default=str)

    def run(folder, cmd, params=None, name=None, host="outlook.office365.com", **kw):
        name = name or cmd[4:]
        key = f"{folder}/{name}"
        try:
            d = auth.invoke(cmd, params, host=host, **kw)
            save(folder, name, d); report["cmdlets"][key] = {"cmdlet": cmd, "count": len(d)}
            log("OK", key, len(d)); return d
        except Exception as x:
            m = str(x)[:400]
            report["cmdlets"][key] = {"cmdlet": cmd, "error": m}
            if any(t in m.lower() for t in ("not recognized", "couldn't be found", "is not available", "not allowed", "isn't supported", "cmdlet")):
                report["not_possible"][key] = m
            log("ERROR", key, m); return None

    def per_object(folder, cmd, items, key_of, param="Identity", name=None, extra=None):
        name = name or cmd[4:]; res, errors = {}, {}
        for it in items:
            k = key_of(it)
            try:
                prm = {param: k}; prm.update(extra or {})
                res[k] = auth.invoke(cmd, prm)
            except Exception as x:
                errors[k] = str(x)[:300]
        save(folder, name, {"data": res, "errors": errors})
        key = f"{folder}/{name}"
        report["cmdlets"][key] = {"cmdlet": cmd, "objects_queried": len(items), "count": sum(len(v) for v in res.values()), "error_count": len(errors)}
        if errors:
            report["cmdlets"][key]["error_example"] = list(errors.values())[0]
        log("PER", key, len(items), "->", report["cmdlets"][key]["count"], "errors", len(errors))
        return res

    X = "exchange"
    if args.retry_gzip:
        old = json.load(open(report_path))
        report.update({k: old[k] for k in ("cmdlets", "not_possible", "start")})
        for k, v in list(old["cmdlets"].items()):
            if "gzip" in v.get("error", ""):
                report["not_possible"].pop(k, None)
                run(X, v["cmdlet"], name=k.split("/")[1])
        json.dump(report, open(report_path, "w"), indent=1, ensure_ascii=False)
        return 0
    for c in dict.fromkeys(SIMPLE):
        run(X, c)
    # Besondere Aufrufe
    run(X, "Get-TenantAllowBlockListItems", {"ListType": "Sender"}, name="TenantAllowBlockListItems_Sender")
    for lt in ("Url", "FileHash", "IP"):
        run(X, "Get-TenantAllowBlockListItems", {"ListType": lt}, name=f"TenantAllowBlockListItems_{lt}")
    run(X, "Get-TenantAllowBlockListSpoofItems", name="TenantAllowBlockListSpoofItems")
    run(X, "Get-MobileDevice", name="MobileDevice")
    run(X, "Get-DistributionGroup", name="DistributionGroup")
    run(X, "Get-Recipient", name="Recipient")
    mailboxes = run(X, "Get-Mailbox", {"ResultSize": "Unlimited", "IncludeInactiveMailbox": True}, name="Mailbox") \
        or run(X, "Get-Mailbox", name="Mailbox") or []
    run(X, "Get-Mailbox", {"ResultSize": "Unlimited", "GroupMailbox": True}, name="Mailbox_GroupMailboxes")
    run(X, "Get-Mailbox", {"ResultSize": "Unlimited", "Archive": True}, name="Mailbox_WithArchive")
    run(X, "Get-CASMailbox", {"ResultSize": "Unlimited"}, name="CASMailbox")
    mailbox_key = lambda m: m.get("ExternalDirectoryObjectId") or m.get("Guid") or m.get("PrimarySmtpAddress")
    people = [m for m in mailboxes if not m.get("IsInactiveMailbox")] or mailboxes
    if args.max_mailboxes:
        people = people[:args.max_mailboxes]
    per_object(X, "Get-MailboxPermission", people, mailbox_key)
    per_object(X, "Get-RecipientPermission", people, mailbox_key, param="Identity")
    per_object(X, "Get-InboxRule", people, mailbox_key, param="Mailbox", extra={"IncludeHidden": True})
    per_object(X, "Get-MailboxAutoReplyConfiguration", people, mailbox_key)
    per_object(X, "Get-MailboxRegionalConfiguration", people, mailbox_key)

    def load(name):
        p_ = os.path.join(out, X, name + ".json")
        return json.load(open(p_)) if os.path.exists(p_) else []
    group_key = lambda g: g.get("ExternalDirectoryObjectId") or g.get("Guid")
    per_object(X, "Get-DistributionGroupMember", load("DistributionGroup"), group_key, name="DistributionGroupMember")
    per_object(X, "Get-RoleGroupMember", load("RoleGroup"), lambda g: g.get("Name"), name="RoleGroupMember")
    for lt in ("Members", "Owners"):
        per_object(X, "Get-UnifiedGroupLinks", load("UnifiedGroup"), group_key, name="UnifiedGroupLinks_" + lt, extra={"LinkType": lt})
    json.dump(report, open(report_path, "w"), indent=1, ensure_ascii=False)
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/file_versions.pyPython · 5,9 kB
"""Dateiversionen aller Laufwerke (nur GET), wiederaufnehmbar.

Pro Datei: Metadaten aller Versionen nach versions/_meta/<drive>/<item>.json. Ältere Versionen
(ohne die aktuelle, das ist der erste Eintrag; höchstens --max-versions neueste ältere) werden als Inhalt
nach versions/<drive>/<item-id>/<version> geladen. Das Manifest ist versions_manifest.jsonl."""
import hashlib, json, os, re, sys, threading, time
from concurrent.futures import ThreadPoolExecutor
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth


def sanitize(s):
    return re.sub(r'[^A-Za-z0-9._-]', "_", s)


def main(argv=None):
    p = auth.make_parser("Download file versions of all drives (read-only)")
    p.add_argument("--max-versions", type=int, default=10, help="older versions per file (default 10)")
    p.add_argument("--max-files", type=int, default=0, help="limit the number of files (0 = all)")
    p.add_argument("--threads", type=int, default=int(os.environ.get("THREADS", "6")))
    args = p.parse_args(argv)
    import requests
    out, max_v = args.out, args.max_versions
    vdir = os.path.join(out, "versions"); os.makedirs(os.path.join(vdir, "_meta"), exist_ok=True)
    manifest = os.path.join(out, "versions_manifest.jsonl")
    lock = threading.Lock(); done = set()
    if os.path.exists(manifest):
        for line in open(manifest):
            try:
                j = json.loads(line); done.add((j["drive"], j["item"], j["version"]))
            except Exception:
                pass
    inventory = json.load(open(os.path.join(out, "graph", "drives_inventory.json")))
    tasks = [(did, it["id"], it["name"]) for did, dr in inventory.items() for it in dr["items"] if "file" in it]
    if args.max_files:
        tasks = tasks[:args.max_files]
    print(len(tasks), "files", flush=True)
    st = {"files": 0, "with_older": 0, "versions_meta": 0, "downloaded": 0, "bytes": 0, "skipped_limit": 0, "errors": 0}
    errors = []

    def meta_error(t, msg):
        with lock:
            st["errors"] += 1; errors.append({"drive": t[0], "item": t[1], "name": t[2], "error": msg})

    def work(t):
        did, iid, name = t; dd = sanitize(did)
        mp = f"{vdir}/_meta/{dd}/{sanitize(iid)}.json"
        if os.path.exists(mp):
            versions = json.load(open(mp))["versions"]
        else:
            versions, url = [], f"/drives/{did}/items/{iid}/versions"
            while url:
                r = auth.g("GET", url)
                if r.status_code != 200:
                    return meta_error(t, f"versions {r.status_code} {r.text[:120]}")
                j = r.json(); versions += j.get("value", []); url = j.get("@odata.nextLink")
            os.makedirs(os.path.dirname(mp), exist_ok=True)
            json.dump({"drive": did, "item": iid, "name": name, "versions": versions}, open(mp, "w"), ensure_ascii=False)
        with lock:
            st["files"] += 1; st["versions_meta"] += len(versions)
        older = versions[1:]
        if older:
            with lock:
                st["with_older"] += 1
        if len(older) > max_v:
            with lock:
                st["skipped_limit"] += len(older) - max_v
        for v in older[:max_v]:
            vid = v["id"]
            if (did, iid, vid) in done:
                continue
            target = f"{vdir}/{dd}/{sanitize(iid)}/{sanitize(vid)}"
            ok, err = False, ""
            for attempt in range(5):
                try:
                    r = auth.g("GET", f"/drives/{did}/items/{iid}/versions/{vid}/content", allow_redirects=False)
                    if r.status_code in (301, 302, 303, 307):
                        url = r.headers["Location"]
                    elif r.status_code == 200:
                        url = None
                    else:
                        err = f"content {r.status_code}"; break
                    os.makedirs(os.path.dirname(target), exist_ok=True); h = hashlib.sha256()
                    if url is None:
                        c = r.content; open(target + ".part", "wb").write(c); h.update(c)
                    else:
                        with requests.get(url, stream=True, timeout=120) as s:
                            if s.status_code in (429, 500, 502, 503, 504):
                                time.sleep(int(s.headers.get("Retry-After", 5 * (attempt + 1)))); continue
                            s.raise_for_status()
                            with open(target + ".part", "wb") as f:
                                for c in s.iter_content(1 << 20):
                                    f.write(c); h.update(c)
                    os.replace(target + ".part", target); ok = True; break
                except Exception as e:
                    err = str(e)[:120]; time.sleep(3 * (attempt + 1))
            if ok:
                rec = {"drive": did, "item": iid, "name": name, "version": vid, "size": os.path.getsize(target),
                       "sha256": h.hexdigest(), "lastModifiedDateTime": v.get("lastModifiedDateTime"),
                       "lastModifiedBy": (v.get("lastModifiedBy") or {}).get("user", {}).get("displayName"),
                       "path": f"versions/{dd}/{sanitize(iid)}/{sanitize(vid)}"}
                with lock:
                    open(manifest, "a").write(json.dumps(rec, ensure_ascii=False) + "\n")
                    st["downloaded"] += 1; st["bytes"] += rec["size"]
            else:
                meta_error(t, f"version {vid}: {err}")
    counter = [0]

    def work_counted(t):
        work(t)
        with lock:
            counter[0] += 1
            if counter[0] % 100 == 0:
                print(counter[0], st, flush=True)
    with ThreadPoolExecutor(args.threads) as ex:
        list(ex.map(work_counted, tasks))
    auth.write_json(os.path.join(out, "versions_errors.json"), errors)
    auth.write_json(os.path.join(out, "versions_status.json"), {"max_older_versions_per_file": max_v, **st})
    print("DONE", st)
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/files_export.pyPython · 4,7 kB
"""Lädt alle Dateien der SharePoint-/OneDrive-Laufwerke herunter (nur GET).
Quelle ist graph/drives_inventory.json (siehe drive_inventory.py). Wiederaufnehmbar: Dateien mit
passender Grösse und Manifest-Eintrag werden übersprungen. Schreibt files/, manifest.jsonl und files_errors.json."""
import hashlib, json, os, re, sys, threading, time
from concurrent.futures import ThreadPoolExecutor
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth


def sanitize(s):
    return re.sub(r'[\x00-\x1f<>:"|?*\\]', "_", s).strip() or "_"


def main(argv=None):
    p = auth.make_parser("Download all drive files (read-only)")
    p.add_argument("--max-files", type=int, default=0, help="limit the number of files (0 = all)")
    p.add_argument("--threads", type=int, default=4)
    args = p.parse_args(argv)
    import requests
    out = args.out
    inventory = json.load(open(os.path.join(out, "graph", "drives_inventory.json")))
    files_dir = os.path.join(out, "files"); os.makedirs(files_dir, exist_ok=True)
    manifest = os.path.join(out, "manifest.jsonl")
    lock = threading.Lock()
    done = {}
    if os.path.exists(manifest):
        for line in open(manifest):
            try:
                j = json.loads(line); done[j["path"]] = j
            except Exception:
                pass
    tasks, errors = [], []
    for drive_id, dr in inventory.items():
        if not isinstance(dr["items"], list):
            errors.append((drive_id, dr["name"], "inventory faulty")); continue
        items = {i["id"]: i for i in dr["items"]}
        root = next((i["id"] for i in dr["items"] if i.get("name") == "root" and "folder" in i), None)
        m = re.search(r"https://[^/]+/(?:sites|teams|personal)/([^/]+)", dr.get("webUrl") or "")
        site = sanitize(m.group(1) if m else (dr.get("webUrl") or drive_id)[:40])
        lib = sanitize(dr.get("name") or drive_id[-8:])

        def rel_path(i, items=items, root=root):
            parts = []
            while i and i in items and i != root:
                parts.append(sanitize(items[i]["name"])); i = items[i].get("parentReference", {}).get("id")
            return "/".join(reversed(parts))
        for it in dr["items"]:
            if "file" in it and not it.get("deleted"):
                tasks.append((drive_id, it, f"{site}/{lib}/{rel_path(it['id'])}"))
    if args.max_files:
        tasks = tasks[:args.max_files]
    print(len(tasks), "files", sum(t[1].get("size", 0) for t in tasks) / 1e9, "GB", flush=True)
    counter = [0, 0]

    def work(t):
        drive_id, it, rel = t
        target = os.path.join(files_dir, rel)
        if rel in done and os.path.exists(target) and os.path.getsize(target) == it.get("size"):
            return
        os.makedirs(os.path.dirname(target), exist_ok=True)
        err = ""
        for attempt in range(6):
            try:
                r = auth.g("GET", f"/drives/{drive_id}/items/{it['id']}")
                if r.status_code in (429, 500, 502, 503, 504):
                    time.sleep(int(r.headers.get("Retry-After", 5 * (attempt + 1)))); continue
                if r.status_code != 200:
                    raise Exception(f"meta {r.status_code}")
                url = r.json().get("@microsoft.graph.downloadUrl")
                if not url:
                    raise Exception("no downloadUrl")
                h = hashlib.sha256()
                with requests.get(url, stream=True, timeout=120) as s:
                    if s.status_code in (429, 500, 502, 503, 504):
                        time.sleep(int(s.headers.get("Retry-After", 5 * (attempt + 1)))); continue
                    s.raise_for_status()
                    with open(target + ".part", "wb") as f:
                        for c in s.iter_content(1 << 20):
                            f.write(c); h.update(c)
                os.replace(target + ".part", target)
                rec = {"path": rel, "size": os.path.getsize(target), "sha256": h.hexdigest(),
                       "lastModified": it.get("lastModifiedDateTime"), "webUrl": it.get("webUrl")}
                with lock:
                    open(manifest, "a").write(json.dumps(rec, ensure_ascii=False) + "\n"); counter[0] += 1
                    if counter[0] % 50 == 0:
                        print(counter, flush=True)
                return
            except Exception as e:
                err = str(e)[:150]; time.sleep(3 * (attempt + 1))
        with lock:
            counter[1] += 1; errors.append((rel, err))
    with ThreadPoolExecutor(args.threads) as ex:
        list(ex.map(work, tasks))
    auth.write_json(os.path.join(out, "files_errors.json"), errors)
    print("DONE", counter, len(errors))
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/graph_export.pyPython · 7,8 kB
"""Graph-Export (nur lesend) eines Tenants als JSON: Organisation, Benutzer, Gruppen, Richtlinien,
Rollen, Apps, Intune und SharePoint-Sites mit Listen. Alle Aufrufe sind GET.

Beispiel:  python3 graph_export.py --out ./backup/contoso/20261116-0557
"""
import json, os, sys, time
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth


def main(argv=None):
    p = auth.make_parser("Read-only Graph export of a tenant as JSON")
    p.add_argument("--max-sites", type=int, default=0, help="limit the number of SharePoint sites (0 = all)")
    args = p.parse_args(argv)
    out = os.path.join(args.out, "graph")   # alle weiteren Skripte lesen aus <out>/graph/
    os.makedirs(out, exist_ok=True)
    log_file = open(os.path.join(out, "export.log"), "a")
    status = {}

    def log(s):
        log_file.write(time.strftime("%H:%M:%S ") + s + "\n"); log_file.flush(); print(s)

    def fetch(name, path, beta=False):
        data, error = auth.get_all(path, beta)
        if error:
            status[name] = {"ok": False, "error": error}; log(f"MISSING {name}: {error[:120]}"); return None
        auth.write_json(os.path.join(out, name + ".json"), data)
        status[name] = {"ok": True, "count": len(data)}; log(f"ok {name}: {len(data)}"); return data

    fetch("organization", "/organization")
    fetch("domains", "/domains")
    fetch("subscribedSkus", "/subscribedSkus")
    fetch("directory_subscriptions", "/directory/subscriptions")
    users = fetch("users", "/users?$top=999&$select=id,displayName,userPrincipalName,mail,givenName,surname,jobTitle,department,officeLocation,city,country,usageLocation,mobilePhone,businessPhones,accountEnabled,assignedLicenses,userType,companyName,employeeId,onPremisesSyncEnabled,createdDateTime,preferredLanguage,streetAddress,postalCode,state,proxyAddresses,otherMails")
    if users:
        managers = {}
        for u in users:
            r = auth.g("GET", f"/users/{u['id']}/manager?$select=id,userPrincipalName")
            if r.status_code == 200:
                managers[u["userPrincipalName"]] = r.json().get("userPrincipalName")
        auth.write_json(os.path.join(out, "users_manager.json"), managers)
        status["users_manager"] = {"ok": True, "count": len(managers)}
        member_of = {}
        for u in users:
            r = auth.g("GET", f"/users/{u['id']}/memberOf?$select=id,displayName")
            if r.status_code == 200:
                member_of[u["userPrincipalName"]] = [(x.get("@odata.type"), x.get("id"), x.get("displayName"))
                                                     for x in r.json().get("value", [])]
            else:
                member_of = None; log(f"memberOf: {r.status_code}"); break
        if member_of is not None:
            auth.write_json(os.path.join(out, "users_member_of.json"), member_of)
            status["users_member_of"] = {"ok": True, "count": len(member_of)}
        else:
            status["users_member_of"] = {"ok": False, "error": "403"}
    groups = fetch("groups", "/groups?$top=999")
    if groups:
        members = {}
        for gr in groups:
            m, e1 = auth.get_all(f"/groups/{gr['id']}/members?$select=id,displayName,userPrincipalName")
            o, e2 = auth.get_all(f"/groups/{gr['id']}/owners?$select=id,displayName,userPrincipalName")
            members[gr["id"]] = {"displayName": gr.get("displayName"), "members": m, "owners": o, "error": e1 or e2}
        auth.write_json(os.path.join(out, "groups_members.json"), members)
        status["groups_members"] = {"ok": True, "count": len(members)}
    for name, path, beta in [
        ("conditionalAccess_policies", "/identity/conditionalAccess/policies", False),
        ("conditionalAccess_namedLocations", "/identity/conditionalAccess/namedLocations", False),
        ("conditionalAccess_authenticationStrengths", "/identity/conditionalAccess/authenticationStrength/policies", False),
        ("directoryRoles", "/directoryRoles?$expand=members", False),
        ("roleDefinitions", "/roleManagement/directory/roleDefinitions", False),
        ("roleAssignments", "/roleManagement/directory/roleAssignments", False),
        ("roleEligibilitySchedules", "/roleManagement/directory/roleEligibilitySchedules", False),
        ("applications", "/applications?$top=999", False),
        ("servicePrincipals", "/servicePrincipals?$top=999", False),
        ("oauth2PermissionGrants", "/oauth2PermissionGrants", False),
        ("administrativeUnits", "/directory/administrativeUnits", False),
        ("authorizationPolicy", "/policies/authorizationPolicy", False),
        ("authenticationMethodsPolicy", "/policies/authenticationMethodsPolicy", False),
        ("identitySecurityDefaults", "/policies/identitySecurityDefaultsEnforcementPolicy", False),
        ("crossTenantAccessPolicy", "/policies/crossTenantAccessPolicy", False),
        ("tokenLifetimePolicies", "/policies/tokenLifetimePolicies", False),
        ("groupSettings", "/groupSettings", False),
        ("organizationBranding", "/organization/{org}/branding", False),
        ("intune_deviceConfigurations", "/deviceManagement/deviceConfigurations", False),
        ("intune_deviceCompliancePolicies", "/deviceManagement/deviceCompliancePolicies", False),
        ("intune_configurationPolicies", "/deviceManagement/configurationPolicies", True),
        ("intune_groupPolicyConfigurations", "/deviceManagement/groupPolicyConfigurations", True),
        ("intune_mobileApps", "/deviceAppManagement/mobileApps", False),
        ("intune_appProtectionPolicies", "/deviceAppManagement/managedAppPolicies", False),
        ("intune_managedDevices", "/deviceManagement/managedDevices", False),
        ("devices", "/devices?$top=999", False),
        ("teams", "/teams", False),
        ("sites", "/sites/getAllSites?$top=999", False),
    ]:
        if "{org}" in path:
            try:
                path = path.replace("{org}", json.load(open(os.path.join(out, "organization.json")))[0]["id"])
            except Exception:
                continue
        fetch(name, path, beta)
    sites = None
    try:
        sites = json.load(open(os.path.join(out, "sites.json")))
    except Exception:
        sites = fetch("sites", "/sites?search=*&$top=999")
    if sites and args.max_sites:
        sites = sites[:args.max_sites]
    if sites:
        site_dir = os.path.join(out, "sharepoint")
        os.makedirs(site_dir, exist_ok=True)
        for s in sites:
            sid = s["id"]
            name = (s.get("webUrl", "").rstrip("/").split("/")[-1] or "root").replace("/", "_")[:60]
            info = {"site": s}
            for key, path in [("lists", f"/sites/{sid}/lists?$expand=columns"), ("contentTypes", f"/sites/{sid}/contentTypes"),
                              ("columns", f"/sites/{sid}/columns"), ("drives", f"/sites/{sid}/drives"),
                              ("permissions", f"/sites/{sid}/permissions"), ("pages", f"/sites/{sid}/pages")]:
                d, e = auth.get_all(path, beta=(key == "pages"))
                info[key] = d if not e else {"error": e}
            # Listeneinträge (Felder) je Liste, ausser Systemlisten
            items = {}
            for lst in (info["lists"] if isinstance(info["lists"], list) else []):
                if lst.get("list", {}).get("hidden"):
                    continue
                d, e = auth.get_all(f"/sites/{sid}/lists/{lst['id']}/items?$expand=fields&$top=999")
                items[lst.get("displayName")] = d if not e else {"error": e}
            info["items"] = items
            auth.write_json(os.path.join(site_dir, f"{name}-{sid.split(',')[1][:8]}.json"), info)
        status["sharepoint_sites_detail"] = {"ok": True, "count": len(sites)}
        log(f"ok sharepoint detail: {len(sites)}")
    auth.write_json(os.path.join(out, "_status.json"), status)
    log("done")
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/graph_settings.pyPython · 10,5 kB
"""graph_settings: Richtlinien, Rollen, Intune, Verwaltungseinheiten und Lizenzen. Nur GET.
Braucht graph/users.json. Ausgabe nach graph_settings/; Endpunkte ohne Zugriff stehen in not_possible.json."""
import os, sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth
from m365_auth import get_all, write_json


def main(argv=None):
    p = auth.make_parser("Export policies, roles, Intune, administrative units and licenses (read-only)")
    args = p.parse_args(argv)
    out = args.out
    root = os.path.join(out, "graph_settings")
    not_possible, ok = {}, {}

    def fetch(name, url, beta=False):
        d, f = get_all(url, beta)
        if f:
            not_possible[name] = {"url": url, "beta": beta, "status": int(f.split()[0]), "message": f[:250]}
            auth.log(out, f"MISSING {name}: {f[:120]}", "graph_settings")
            if not d:
                return None
        write_json(os.path.join(root, name + ".json"), d); ok[name] = len(d)
        auth.log(out, f"ok {name}: {len(d)}", "graph_settings"); return d

    def per_item(name, items, template, beta=False, field="id"):
        res = {}
        for it in items or []:
            d, f = get_all(template.format(id=it[field]), beta)
            res[it[field]] = d if not f else {"error": f[:200]}
            if f:
                not_possible[f"{name}:{it[field]}"] = {"status": int(f.split()[0]), "message": f[:200]}
        write_json(os.path.join(root, name + ".json"), res); ok[name] = len(res)

    os.makedirs(root, exist_ok=True)
    # Conditional Access und Richtlinien
    fetch("conditionalAccess_policies", "/identity/conditionalAccess/policies")
    fetch("conditionalAccess_policies_beta", "/identity/conditionalAccess/policies", True)
    fetch("conditionalAccess_namedLocations", "/identity/conditionalAccess/namedLocations")
    fetch("conditionalAccess_authenticationStrengths", "/identity/conditionalAccess/authenticationStrength/policies")
    fetch("conditionalAccess_authenticationContextClassReferences", "/identity/conditionalAccess/authenticationContextClassReferences")
    fetch("conditionalAccess_templates", "/identity/conditionalAccess/templates")
    fetch("authenticationMethodsPolicy", "/policies/authenticationMethodsPolicy")
    fetch("authenticationMethodsPolicy_beta", "/policies/authenticationMethodsPolicy", True)
    fetch("authenticationMethods_registrationReport", "/reports/authenticationMethods/userRegistrationDetails")
    for n, path in [("identitySecurityDefaultsEnforcementPolicy", "identitySecurityDefaultsEnforcementPolicy"), ("authorizationPolicy", "authorizationPolicy"),
                    ("adminConsentRequestPolicy", "adminConsentRequestPolicy"), ("permissionGrantPolicies", "permissionGrantPolicies"),
                    ("activityBasedTimeoutPolicies", "activityBasedTimeoutPolicies"), ("claimsMappingPolicies", "claimsMappingPolicies"),
                    ("homeRealmDiscoveryPolicies", "homeRealmDiscoveryPolicies"), ("tokenIssuancePolicies", "tokenIssuancePolicies"),
                    ("tokenLifetimePolicies", "tokenLifetimePolicies"), ("featureRolloutPolicies", "featureRolloutPolicies"),
                    ("crossTenantAccessPolicy", "crossTenantAccessPolicy"), ("crossTenantAccessPolicy_default", "crossTenantAccessPolicy/default"),
                    ("crossTenantAccessPolicy_partners", "crossTenantAccessPolicy/partners"), ("externalIdentitiesPolicy", "externalIdentitiesPolicy"),
                    ("b2cAuthenticationMethodsPolicy", "b2cAuthenticationMethodsPolicy"), ("appManagementPolicies", "appManagementPolicies"),
                    ("servicePrincipalCreationPolicies", "servicePrincipalCreationPolicies"),
                    ("roleManagementPolicies", "roleManagementPolicies?$filter=scopeId eq '/' and scopeType eq 'DirectoryRole'"),
                    ("roleManagementPolicyAssignments", "roleManagementPolicyAssignments?$filter=scopeId eq '/' and scopeType eq 'DirectoryRole'"),
                    ("deviceRegistrationPolicy", "deviceRegistrationPolicy"), ("directoryRoleAccessReviewPolicy", "directoryRoleAccessReviewPolicy"),
                    ("identityProviders", "/identity/identityProviders"), ("authenticationFlowsPolicy", "authenticationFlowsPolicy")]:
        fetch("policies_" + n, path if path.startswith("/") else "/policies/" + path)
    # beta ergänzend nur für einige
    for n, path in [("beta_policies_deviceRegistrationPolicy", "/policies/deviceRegistrationPolicy"), ("beta_policies_crossTenantAccessPolicy", "/policies/crossTenantAccessPolicy"),
                    ("beta_policies_mobilityManagementPolicies", "/policies/mobileDeviceManagementPolicies"), ("beta_policies_accessReviewPolicy", "/policies/accessReviewPolicy"),
                    ("beta_policies_activityBasedTimeoutPolicies", "/policies/activityBasedTimeoutPolicies"), ("beta_policies_identitySecurityDefaults", "/policies/identitySecurityDefaultsEnforcementPolicy")]:
        fetch(n, path, True)
    fetch("organization_settings", "/organization?$select=id,displayName,verifiedDomains,technicalNotificationMails,securityComplianceNotificationMails,privacyProfile,onPremisesSyncEnabled")
    fetch("directorySettingTemplates", "/directorySettingTemplates")
    fetch("domains", "/domains")
    fetch("domainNameReferences_none", "/domains?$select=id,authenticationType,isDefault,isVerified,supportedServices")
    # Rollen
    fetch("roleDefinitions", "/roleManagement/directory/roleDefinitions")
    fetch("roleAssignments", "/roleManagement/directory/roleAssignments?$expand=principal")
    for n in ("roleEligibilitySchedules", "roleAssignmentSchedules", "roleEligibilityScheduleInstances",
              "roleAssignmentScheduleInstances", "roleEligibilityScheduleRequests", "roleAssignmentScheduleRequests"):
        fetch(n, "/roleManagement/directory/" + n)
    dr = fetch("directoryRoles", "/directoryRoles")
    per_item("directoryRoles_members", dr, "/directoryRoles/{id}/members?$select=id,displayName,userPrincipalName")
    fetch("directoryRoleTemplates", "/directoryRoleTemplates")
    fetch("intune_roleDefinitions_rbac", "/deviceManagement/roleDefinitions")
    # Verwaltungseinheiten
    au = fetch("administrativeUnits", "/directory/administrativeUnits")
    per_item("administrativeUnits_members", au, "/directory/administrativeUnits/{id}/members?$select=id,displayName")
    per_item("administrativeUnits_scopedRoleMembers", au, "/directory/administrativeUnits/{id}/scopedRoleMembers")
    # Lizenzen
    users = auth.read_users(out)
    fetch("subscribedSkus", "/subscribedSkus")
    per_item("licenses_per_user", users, "/users/{id}/licenseDetails")
    write_json(os.path.join(root, "license_assignments_assignedLicenses.json"),
               {u["userPrincipalName"]: u.get("assignedLicenses") for u in users})
    per_item("authentication_methods_per_user", users, "/users/{id}/authentication/methods")
    fetch("conditionalAccess_none_placeholder_check", "/identity/conditionalAccess/policies?$top=1")

    # Intune
    def intune(name, url, beta=False, sub=None, expand=None):
        d = fetch("intune_" + name, url + (f"?$expand={expand}" if expand else ""), beta)
        if d is None and expand:
            d = fetch("intune_" + name, url, beta)
        if d and sub:
            for s in sub:
                per_item(f"intune_{name}_{s}", d, url + "/{id}/" + s, beta)
        return d
    intune("deviceConfigurations", "/deviceManagement/deviceConfigurations", sub=["assignments", "deviceStatusOverview"])
    intune("deviceConfigurations_beta", "/deviceManagement/deviceConfigurations", True, sub=["assignments"])
    intune("deviceCompliancePolicies", "/deviceManagement/deviceCompliancePolicies", sub=["assignments", "scheduledActionsForRule"])
    intune("configurationPolicies", "/deviceManagement/configurationPolicies", True, sub=["settings", "assignments"])
    intune("groupPolicyConfigurations", "/deviceManagement/groupPolicyConfigurations", True, sub=["assignments", "definitionValues"])
    intune("mobileApps", "/deviceAppManagement/mobileApps", sub=["assignments"])
    intune("managedAppPolicies", "/deviceAppManagement/managedAppPolicies")
    intune("androidManagedAppProtections", "/deviceAppManagement/androidManagedAppProtections", sub=["assignments", "apps"])
    intune("iosManagedAppProtections", "/deviceAppManagement/iosManagedAppProtections", sub=["assignments", "apps"])
    intune("targetedManagedAppConfigurations", "/deviceAppManagement/targetedManagedAppConfigurations", sub=["assignments"])
    intune("windowsInformationProtectionPolicies", "/deviceAppManagement/windowsInformationProtectionPolicies", sub=["assignments"])
    intune("mdmWindowsInformationProtectionPolicies", "/deviceAppManagement/mdmWindowsInformationProtectionPolicies", sub=["assignments"])
    intune("deviceEnrollmentConfigurations", "/deviceManagement/deviceEnrollmentConfigurations", sub=["assignments"])
    intune("windowsAutopilotDeploymentProfiles", "/deviceManagement/windowsAutopilotDeploymentProfiles", sub=["assignments"])
    intune("deviceManagementScripts", "/deviceManagement/deviceManagementScripts", sub=["assignments"])
    intune("deviceHealthScripts", "/deviceManagement/deviceHealthScripts", True, sub=["assignments"])
    intune("deviceShellScripts", "/deviceManagement/deviceShellScripts", True, sub=["assignments"])
    intune("intents", "/deviceManagement/intents", True, sub=["settings", "assignments"])
    intune("managedDevices", "/deviceManagement/managedDevices")
    intune("deviceCategories", "/deviceManagement/deviceCategories")
    intune("deviceManagementSettings", "/deviceManagement")
    intune("termsAndConditions", "/deviceManagement/termsAndConditions", sub=["assignments"])
    intune("notificationMessageTemplates", "/deviceManagement/notificationMessageTemplates")
    intune("assignmentFilters", "/deviceManagement/assignmentFilters", True)
    intune("mobileAppCategories", "/deviceAppManagement/mobileAppCategories")
    intune("managedEBooks", "/deviceAppManagement/managedEBooks")
    intune("windowsFeatureUpdateProfiles", "/deviceManagement/windowsFeatureUpdateProfiles", True, sub=["assignments"])
    intune("windowsQualityUpdateProfiles", "/deviceManagement/windowsQualityUpdateProfiles", True, sub=["assignments"])
    intune("templates", "/deviceManagement/templates", True)
    write_json(os.path.join(root, "_summary.json"), {"ok": ok, "not_possible": not_possible})
    write_json(os.path.join(root, "not_possible.json"), not_possible)
    auth.log(out, f"done {len(ok)} ok, {len(not_possible)} not possible", "graph_settings")
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/mailboxes.pyPython · 6,2 kB
"""Postfächer: Mail (mit Anhängen), Kalender und Kontakte je Benutzer. Nur GET, wiederaufnehmbar.
Braucht graph/users.json (siehe graph_export.py). Ausgabe nach mailboxes/<upn>/."""
import json, os, sys
from concurrent.futures import ThreadPoolExecutor
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth
from m365_auth import get_all, safe_name, write_json


def folders_recursive(uid, url):
    res, f = get_all(f"{url}?$top=100&includeHiddenFolders=true")
    if f:
        return res, f
    out = []
    for o in res:
        out.append(o)
        if o.get("childFolderCount", 0):
            kids, f2 = folders_recursive(uid, f"/users/{uid}/mailFolders/{o['id']}/childFolders")
            if f2:
                return out, f2
            out += kids
    return out, None


def attachments(uid, m, mdir):
    ad = os.path.join(mdir, "attachments", safe_name(m["id"][-24:]))
    lst, f = get_all(f"/users/{uid}/messages/{m['id']}/attachments?$select=id,name,size,contentType,isInline,lastModifiedDateTime")
    if f:
        return {"error": f}
    meta = []
    for a in lst:
        target = os.path.join(ad, safe_name(a["id"][-12:]) + "_" + safe_name(a.get("name")))
        if not os.path.exists(target):
            r = auth.g("GET", f"/users/{uid}/messages/{m['id']}/attachments/{a['id']}/$value")
            if r.status_code == 200:
                os.makedirs(ad, exist_ok=True); open(target, "wb").write(r.content)
            else:
                a["error"] = f"{r.status_code} {r.text[:150]}"
                r2 = auth.g("GET", f"/users/{uid}/messages/{m['id']}/attachments/{a['id']}?$expand=microsoft.graph.itemattachment/item")
                if r2.status_code == 200:
                    write_json(target + ".json", r2.json()); a["fallback_json"] = True
        a["file"] = os.path.relpath(target, mdir); meta.append(a)
    return meta


def backup_user(u, root, out):
    uid = u["id"]; upn = u["userPrincipalName"]; d = os.path.join(root, safe_name(upn))
    done_file = os.path.join(d, "done.json")
    if os.path.exists(done_file):
        return json.load(open(done_file))
    r = auth.g("GET", f"/users/{uid}/mailFolders?$top=1")
    if r.status_code != 200:
        st = {"upn": upn, "mailbox": False, "status": r.status_code, "message": r.text[:200]}
        auth.log(out, f"{upn}: no mailbox {r.status_code}", "mail"); write_json(done_file, st); return st
    st = {"upn": upn, "mailbox": True, "errors": [], "folders": 0, "expected": 0, "messages": 0,
          "attachments": 0, "events": 0, "contacts": 0}
    folders, f = folders_recursive(uid, f"/users/{uid}/mailFolders")
    if f:
        st["errors"].append("folders " + f)
    write_json(os.path.join(d, "mail/folders.json"), folders); st["folders"] = len(folders)
    for i, fo in enumerate(folders):
        fd = os.path.join(d, "mail", f"{i:03d}_{safe_name(fo.get('displayName'))}")
        st["expected"] += fo.get("totalItemCount", 0)
        pf = os.path.join(fd, "messages.json")
        if os.path.exists(pf):
            msgs = json.load(open(pf))
        else:
            msgs, f = get_all(f"/users/{uid}/mailFolders/{fo['id']}/messages?$top=50")
            if f:
                st["errors"].append(f"{fo.get('displayName')} messages {f}")
                auth.log(out, f"{upn} {fo.get('displayName')}: {f[:100]}", "mail")
            for m in msgs:
                if m.get("hasAttachments"):
                    m["_attachments"] = attachments(uid, m, fd)
                    st["attachments"] += len(m["_attachments"]) if isinstance(m["_attachments"], list) else 0
            if not f:
                write_json(pf, msgs)
            else:
                write_json(os.path.join(fd, "messages_incomplete.json"), msgs)
        st["messages"] += len(msgs)
        write_json(os.path.join(fd, "folder.json"), fo)
    # Kalender
    cals, f = get_all(f"/users/{uid}/calendars")
    if f:
        st["errors"].append("calendars " + f)
    write_json(os.path.join(d, "calendar/calendars.json"), cals)
    grp, f = get_all(f"/users/{uid}/calendarGroups")
    if not f:
        write_json(os.path.join(d, "calendar/calendar_groups.json"), grp)
    for i, c in enumerate(cals):
        ev, f = get_all(f"/users/{uid}/calendars/{c['id']}/events?$top=100")
        if f:
            st["errors"].append(f"calendar {c.get('name')} {f}")
        write_json(os.path.join(d, "calendar", f"{i:02d}_{safe_name(c.get('name'))}_events.json"), ev); st["events"] += len(ev)
    # Kontakte
    cf, f = get_all(f"/users/{uid}/contactFolders?$top=100")
    if f:
        st["errors"].append("contact folders " + f); cf = []
    write_json(os.path.join(d, "contacts/contact_folders.json"), cf)
    ct, f = get_all(f"/users/{uid}/contacts?$top=100")
    if f:
        st["errors"].append("contacts " + f)
    write_json(os.path.join(d, "contacts/contacts.json"), ct); st["contacts"] += len(ct)
    for i, c in enumerate(cf):
        ct, f = get_all(f"/users/{uid}/contactFolders/{c['id']}/contacts?$top=100")
        if f:
            st["errors"].append(f"contact folder {c.get('displayName')} {f}")
        write_json(os.path.join(d, "contacts", f"{i:02d}_{safe_name(c.get('displayName'))}.json"), ct); st["contacts"] += len(ct)
    st["deviation"] = st["expected"] - st["messages"]
    auth.log(out, f"{upn}: {st['messages']}/{st['expected']} messages, {st['attachments']} attachments, "
                  f"{st['events']} events, {st['contacts']} contacts, errors {len(st['errors'])}", "mail")
    write_json(done_file, st)
    return st


def main(argv=None):
    p = auth.make_parser("Back up mailboxes, calendars and contacts per user (read-only)")
    p.add_argument("--max-mailboxes", type=int, default=0, help="limit the number of users (0 = all)")
    p.add_argument("--threads", type=int, default=3)
    args = p.parse_args(argv)
    root = os.path.join(args.out, "mailboxes")
    users = [u for u in auth.read_users(args.out) if u.get("userPrincipalName")]
    if args.max_mailboxes:
        users = users[:args.max_mailboxes]
    with ThreadPoolExecutor(args.threads) as ex:
        res = list(ex.map(lambda u: backup_user(u, root, args.out), users))
    write_json(os.path.join(root, "_summary.json"), res)
    auth.log(args.out, "done", "mail")
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/permission_probe.pyPython · 1,2 kB
"""Rechte-Probe: je Bereich ein GET-Aufruf, um zu sehen, welche Berechtigungen die App wirklich hat.
Schreibt permission_probe.json in den Zielordner. Braucht graph/users.json."""
import json, os, sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth


def main(argv=None):
    p = auth.make_parser("Probe the granted permissions with one GET per area")
    args = p.parse_args(argv)
    uid = auth.read_users(args.out)[0]["id"]
    tests = {"mail": f"/users/{uid}/messages?$top=1", "calendar": f"/users/{uid}/events?$top=1", "teams": "/teams",
             "chats": f"/users/{uid}/chats", "conditionalAccess": "/identity/conditionalAccess/policies",
             "intune_compliance": "/deviceManagement/deviceCompliancePolicies"}
    res = {}
    for k, path in tests.items():
        r = auth.g("GET", path)
        code = ""
        try:
            code = (r.json().get("error") or {}).get("code", "")
        except Exception:
            pass
        res[k] = {"path": path.replace(uid, "{userId}"), "status": r.status_code, "error_code": code}
    auth.write_json(os.path.join(args.out, "permission_probe.json"), res)
    print(json.dumps(res, indent=1))
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/report.pyPython · 3,2 kB
"""Sammelbericht über Postfächer, Teams, Graph-Einstellungen und SharePoint-Tenant (nur lokal, kein Netz).
Liest die Zusammenfassungen der anderen Skripte und prüft alle JSON-Dateien auf Lesbarkeit.
Schreibt report.json in den Zielordner."""
import glob, json, os, sys, time
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth


def main(argv=None):
    p = auth.make_parser("Summary report over the backup areas (local only)")
    args = p.parse_args(argv)
    B = args.out
    R = {"created": time.strftime("%Y-%m-%d %H:%M:%S"), "areas": {}}
    # Postfächer
    z = json.load(open(os.path.join(B, "mailboxes/_summary.json")))
    with_mb = [x for x in z if x.get("mailbox")]; without = [x for x in z if not x.get("mailbox")]
    n, b = auth.folder_size(os.path.join(B, "mailboxes"))
    R["areas"]["mailboxes"] = {
        "users_with_mailbox": len(with_mb), "users_without_mailbox": [{"upn": x["upn"], "status": x["status"]} for x in without],
        "messages": sum(x["messages"] for x in with_mb), "expected_totalItemCount": sum(x["expected"] for x in with_mb),
        "attachments": sum(x["attachments"] for x in with_mb), "events": sum(x["events"] for x in with_mb),
        "contacts": sum(x["contacts"] for x in with_mb), "files": n, "bytes": b,
        "errors": [{"upn": x["upn"], "errors": x["errors"]} for x in with_mb if x["errors"]],
        "per_user": [{"upn": x["upn"], "messages": x["messages"], "expected": x["expected"], "deviation": x["deviation"]} for x in with_mb]}
    # Teams
    tz = json.load(open(os.path.join(B, "teams/_summary.json"))); nmt = json.load(open(os.path.join(B, "teams/not_possible.json")))
    n, b = auth.folder_size(os.path.join(B, "teams"))
    R["areas"]["teams"] = {**tz, "files": n, "bytes": b, "not_possible_count": len(nmt),
                           "not_possible_status": sorted({str(v.get("status")) for v in nmt.values()}),
                           "not_possible_example": list(nmt.items())[:3]}
    gz = json.load(open(os.path.join(B, "graph_settings/_summary.json"))); n, b = auth.folder_size(os.path.join(B, "graph_settings"))
    R["areas"]["graph_settings"] = {"ok_datasets": len(gz["ok"]), "objects_total": sum(gz["ok"].values()), "files": n, "bytes": b,
                                    "not_possible": gz["not_possible"]}
    sz = json.load(open(os.path.join(B, "spo_tenant/_summary.json"))); n, b = auth.folder_size(os.path.join(B, "spo_tenant"))
    R["areas"]["spo_tenant"] = {"ok": list(sz["ok"]), "files": n, "bytes": b, "not_possible": sz["not_possible"],
                                "note": "GET REST only (SPO.Tenant, SiteProperties schema); CSOM ProcessQuery/POST deliberately not used (read-only rule)."}
    # JSON-Prüfung
    bad, cnt = [], 0
    for d in ("mailboxes", "teams", "graph_settings", "spo_tenant"):
        for path in glob.glob(os.path.join(B, d, "**/*.json"), recursive=True):
            cnt += 1
            try:
                json.load(open(path))
            except Exception:
                bad.append(path)
    R["json_checked"] = cnt; R["json_broken"] = bad
    auth.write_json(os.path.join(B, "report.json"), R)
    print(cnt, bad[:5])
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/secrets_inventory.pyPython · 1,5 kB
"""Inventar der App-Secrets und -Zertifikate (nur Metadaten, KEINE Werte).
Liest graph/applications.json und graph/servicePrincipals.json, schreibt secrets_inventory.json.
Der Restore (restore/secrets_restore.py) nutzt das Inventar als Gegenprobe."""
import json, os, sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth


def main(argv=None):
    p = auth.make_parser("Inventory of app secrets and certificates (metadata only, no values)")
    args = p.parse_args(argv)
    out = []
    for source, _ in (("applications", "application"), ("servicePrincipals", "servicePrincipal")):
        for a in json.load(open(f"{args.out}/graph/{source}.json")):
            for key, kind in (("passwordCredentials", "Secret"), ("keyCredentials", "Certificate/Key")):
                for c in a.get(key) or []:
                    out.append({"source": source, "app": a.get("displayName"), "appId": a.get("appId"), "objectId": a.get("id"),
                                "kind": kind, "display_name": c.get("displayName"), "keyId": c.get("keyId"),
                                "start": c.get("startDateTime"), "expires": c.get("endDateTime"),
                                "usage": c.get("usage"), "key_type": c.get("type")})
    auth.write_json(args.out + "/secrets_inventory.json", out)
    print(len(out), "entries;", sum(1 for o in out if o["source"] == "applications"), "on applications")
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/sharepoint_lists.pyPython · 7,0 kB
"""Listeneintrag-Versionen, Papierkorb-Metadaten, Seiten mit canvasLayout, Inhaltstypen der Listen,
Site-Pages-Liste und Strukturprüfung (Stichprobe von 5 Sites, live gegen Backup). Nur GET.

Ausgabe: graph/sharepoint/<stem>/{versions/<list>.json, recycle_bin.json, pages_canvas.json,
list_contenttypes.json, termstore.json}, graph/structure_check.json und graph/sharepoint_extended_status.json."""
import glob, json, os, random, re, sys, threading
from concurrent.futures import ThreadPoolExecutor
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth


def sanitize(s):
    return re.sub(r'[^A-Za-z0-9._-]', "_", s)[:80]


def main(argv=None):
    p = auth.make_parser("List item versions, recycle bin, canvas pages and structure check (read-only)")
    p.add_argument("--max-items", type=int, default=1000, help="items per list (default 1000)")
    p.add_argument("--max-sites", type=int, default=0, help="limit the number of sites (0 = all)")
    args = p.parse_args(argv)
    out, max_items = args.out, args.max_items
    site_root = os.path.join(out, "graph", "sharepoint")
    lock = threading.Lock()
    st = {"sites": 0, "lists": 0, "items_with_versions": 0, "versions": 0, "items_skipped_limit": 0,
          "recycle_bin_entries": 0, "canvas_pages": 0, "attachment_items": 0, "errors": []}

    def site_work(f):
        j = json.load(open(f)); site = j["site"]; sid = site["id"]
        stem = os.path.basename(f)[:-5]; d = os.path.join(site_root, stem)
        os.makedirs(d + "/versions", exist_ok=True)
        # Papierkorb (nur Metadaten)
        bin_items, e = auth.get_all(f"/sites/{sid}/recycleBin/items", True)
        auth.write_json(d + "/recycle_bin.json", bin_items if e is None else {"error": e})
        with lock:
            if e:
                st["errors"].append((stem, "recycle_bin", e))
            else:
                st["recycle_bin_entries"] += len(bin_items)
        # Listen: Backup-Listen plus Site Pages
        lists = [(x["id"], x.get("displayName")) for x in (j["lists"] if isinstance(j["lists"], list) else [])]
        r = auth.g("GET", f"/sites/{sid}/lists/Site%20Pages?$select=id,displayName")
        if r.status_code == 200 and r.json()["id"] not in [x[0] for x in lists]:
            lists.append((r.json()["id"], r.json().get("displayName", "Site Pages")))
        content_types = {}
        for lid, lname in lists:
            c, e = auth.get_all(f"/sites/{sid}/lists/{lid}/contentTypes")
            content_types[lname] = c if e is None else {"error": e}
            items, e = auth.get_all(f"/sites/{sid}/lists/{lid}/items?$top=999&$expand=fields")
            if e:
                with lock:
                    st["errors"].append((stem, lname, "items " + e))
                continue
            with lock:
                st["lists"] += 1
                st["attachment_items"] += sum(1 for i in items if (i.get("fields") or {}).get("Attachments") is True)
            res = []
            for i in items[:max_items]:
                v, e = auth.get_all(f"/sites/{sid}/lists/{lid}/items/{i['id']}/versions")
                if e:
                    with lock:
                        st["errors"].append((stem, lname, f"item {i['id']} versions {e}"))
                    continue
                res.append({"itemId": i["id"], "versions": v})
                with lock:
                    st["items_with_versions"] += 1; st["versions"] += len(v)
            if len(items) > max_items:
                with lock:
                    st["items_skipped_limit"] += len(items) - max_items
            auth.write_json(f"{d}/versions/{sanitize(lname)}-{lid[:8]}.json",
                            {"list": lname, "listId": lid, "items_total": len(items), "items_exported": len(res), "items": res})
        auth.write_json(d + "/list_contenttypes.json", content_types)
        # Seiten mit canvasLayout
        pages = j["pages"] if isinstance(j["pages"], list) else []
        canvas = []
        for pg in pages:
            r = auth.g("GET", f"https://graph.microsoft.com/beta/sites/{sid}/pages/{pg['id']}/microsoft.graph.sitePage?$expand=canvasLayout")
            if r.status_code == 200:
                canvas.append(r.json())
            else:
                with lock:
                    st["errors"].append((stem, "page " + pg.get("name", ""), f"canvas {r.status_code}"))
        auth.write_json(d + "/pages_canvas.json", canvas)
        with lock:
            st["canvas_pages"] += len(canvas); st["sites"] += 1; print(st["sites"], stem, flush=True)
        # Termspeicher (nur Versuch)
        r = auth.g("GET", f"/sites/{sid}/termStore/sets")
        if r.status_code == 200:
            auth.write_json(d + "/termstore.json", r.json())
        return stem, r.status_code

    files = sorted(glob.glob(site_root + "/*.json"))
    if args.max_sites:
        files = files[:args.max_sites]
    with ThreadPoolExecutor(3) as ex:
        results = list(ex.map(site_work, files))
    st["termstore_status"] = {}
    for _, c in results:
        st["termstore_status"][c] = st["termstore_status"].get(c, 0) + 1
    # Strukturprüfung: Stichprobe live gegen Backup
    random.seed(5)
    sample = random.sample(files, min(5, len(files))); check = []
    for f in sample:
        j = json.load(open(f)); sid = j["site"]["id"]; entry = {"site": j["site"].get("webUrl")}
        for key, path, beta in [("lists", f"/sites/{sid}/lists", False), ("contentTypes", f"/sites/{sid}/contentTypes", False),
                                ("columns", f"/sites/{sid}/columns", False), ("pages", f"/sites/{sid}/pages", True),
                                ("drives", f"/sites/{sid}/drives", False)]:
            live, er = auth.get_all(path, beta); bk = j[key]
            entry[key] = {"live": None if er else len(live), "backup": len(bk) if isinstance(bk, list) else bk, "error": er}
        per_list = {}
        for lst in (j["lists"] if isinstance(j["lists"], list) else []):
            if lst.get("list", {}).get("hidden"):
                continue
            live, er = auth.get_all(f"/sites/{sid}/lists/{lst['id']}/items?$top=999&$expand=fields")
            bk = j["items"].get(lst["displayName"])
            with_fields = sum(1 for i in (bk or []) if isinstance(i, dict) and i.get("fields")) if isinstance(bk, list) else None
            per_list[lst["displayName"]] = {"live": None if er else len(live), "backup": len(bk) if isinstance(bk, list) else bk,
                                            "backup_with_fields": with_fields, "columns_in_list": len(lst.get("columns", []))}
        entry["list_items"] = per_list
        cvf = os.path.join(site_root, os.path.basename(f)[:-5], "pages_canvas.json")
        entry["canvas_pages_in_backup"] = len(json.load(open(cvf))); check.append(entry)
    auth.write_json(os.path.join(out, "graph", "structure_check.json"), check)
    auth.write_json(os.path.join(out, "graph", "sharepoint_extended_status.json"), st)
    print("DONE", {k: v for k, v in st.items() if k != "errors"}, len(st["errors"]), "errors")
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/spo_tenant.pyPython · 3,4 kB
"""SharePoint-Tenant-Einstellungen über die Admin-Site (REST, nur GET) und Graph (admin/sharepoint/settings).
Das Token für die Admin-Site wird mit dem Zertifikat aus m365_auth geholt; der Host ergibt sich aus
M365_TENANT_NAME (https://<präfix>-admin.sharepoint.com). Ausgabe nach spo_tenant/."""
import os, sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth
from m365_auth import write_json

ENDPOINTS = [
    ("web", "/_api/web"), ("site", "/_api/site"), ("tenant_SPO", "/_api/SPO.Tenant"),
    ("tenant_TenantAdmin", "/_api/Microsoft.Online.SharePoint.TenantAdministration.Tenant"),
    ("tenant_Internal", "/_api/SPOInternalUseOnly.Tenant"), ("tenant_sites", "/_api/SPO.Tenant/GetSitePropertiesFromSharePointByFilters"),
    ("tenantSettings", "/_api/SP.TenantSettings.Current"), ("web_currentuser", "/_api/web/currentuser"), ("web_features", "/_api/web/features"),
    ("web_lists", "/_api/web/lists"), ("site_features", "/_api/site/features"), ("web_roleassignments", "/_api/web/roleassignments"),
    ("siteusers", "/_api/web/siteusers"), ("sitegroups", "/_api/web/sitegroups"), ("webtemplates", "/_api/web/GetAvailableWebTemplates(1033)"),
    ("tenantAppCatalog", "/_api/SP.TenantAdmin.Tenant"), ("hubsites", "/_api/hubsites"), ("themes", "/_api/SP.Utilities.ThemeManager/GetTenantThemingOptions"),
    ("siteproperties_odata", "/_api/Microsoft.Online.SharePoint.TenantAdministration.SiteProperties"),
    ("storage_quota", "/_api/StorageQuotas")]


def sp_get(admin_host, path, timeout=60):
    """GET gegen die SharePoint-REST-Schnittstelle der Admin-Site, mit App-only-Token."""
    import requests
    return requests.get(admin_host + path, timeout=timeout, headers={
        "Authorization": "Bearer " + auth.token(admin_host + "/.default"),
        "Accept": "application/json;odata=nometadata"})


def main(argv=None):
    p = auth.make_parser("Read SharePoint tenant settings via the admin site (read-only)")
    args = p.parse_args(argv)
    root = os.path.join(args.out, "spo_tenant"); os.makedirs(root, exist_ok=True)
    admin_host = f"https://{auth.tenant_prefix()}-admin.sharepoint.com"
    ok, not_possible = {}, {}
    for name, path in ENDPOINTS:
        try:
            r = sp_get(admin_host, path)
        except Exception as e:
            not_possible[name] = {"url": path, "status": None, "message": str(e)[:200]}; continue
        if r.status_code == 200:
            try:
                write_json(os.path.join(root, name + ".json"), r.json()); ok[name] = 1
            except Exception:
                not_possible[name] = {"url": path, "status": 200, "message": "no JSON answer"}
        else:
            not_possible[name] = {"url": path, "status": r.status_code, "message": r.text[:200]}
        print(name, r.status_code)
    # Graph: SharePoint-Tenant-Einstellungen
    r = auth.g("GET", "https://graph.microsoft.com/v1.0/admin/sharepoint/settings")
    if r.status_code == 200:
        write_json(os.path.join(root, "graph_admin_sharepoint_settings.json"), r.json()); ok["graph_settings"] = 1
    else:
        not_possible["graph_admin_sharepoint_settings"] = {"status": r.status_code, "message": r.text[:200]}
    write_json(os.path.join(root, "not_possible.json"), not_possible)
    write_json(os.path.join(root, "_summary.json"), {"ok": ok, "not_possible": not_possible})
    auth.log(args.out, f"done {len(ok)} ok, {len(not_possible)} not possible", "spo")
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/teams.pyPython · 5,5 kB
"""Teams: Teams, Kanäle, Kanalnachrichten mit Antworten und Chats je Benutzer. Nur GET.
Braucht graph/users.json. Ausgabe nach teams/. Was nicht abrufbar ist, steht in teams/not_possible.json."""
import json, os, sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth
from m365_auth import get_all, safe_name, write_json


def main(argv=None):
    p = auth.make_parser("Back up Teams, channels, messages and chats (read-only)")
    p.add_argument("--max-teams", type=int, default=0, help="limit the number of teams (0 = all)")
    args = p.parse_args(argv)
    out = args.out
    root = os.path.join(out, "teams")
    not_possible = {}

    def note(what, url, f):
        status = int(f.split()[0]) if f and f.split()[0].isdigit() else None
        not_possible[what] = {"url": url, "status": status, "message": f}
        auth.log(out, f"not possible {what}: {f[:150]}", "teams")
        write_json(os.path.join(root, "not_possible.json"), not_possible)

    def do_teams():
        ts, f = get_all("/groups?$filter=resourceProvisioningOptions/Any(x:x eq 'Team')&$select=id,displayName,description,visibility,createdDateTime")
        if f:
            note("teams_list", "/groups(Team)", f); ts, f = get_all("/teams")
        if args.max_teams:
            ts = ts[:args.max_teams]
        write_json(os.path.join(root, "teams.json"), ts)
        n = {"teams": len(ts), "channels": 0, "messages": 0, "replies": 0}
        for t in ts:
            td = os.path.join(root, "teams", safe_name(t["displayName"]) + "_" + t["id"][:8])
            if os.path.exists(os.path.join(td, "done")):
                continue
            r = auth.g("GET", f"/teams/{t['id']}")
            if r.status_code == 200:
                write_json(os.path.join(td, "team.json"), r.json())
            else:
                note(f"team {t['displayName']}", f"/teams/{t['id']}", f"{r.status_code} {r.text[:200]}"); continue
            d, f = get_all(f"/teams/{t['id']}/members")
            if f:
                note(f"{t['displayName']} members", "members", f)
            else:
                write_json(os.path.join(td, "members.json"), d)
            chs, f = get_all(f"/teams/{t['id']}/channels")
            if f:
                note(f"{t['displayName']} channels", "channels", f); continue
            write_json(os.path.join(td, "channels.json"), chs); n["channels"] += len(chs)
            for c in chs:
                cd = os.path.join(td, "channel_" + safe_name(c["displayName"]) + "_" + c["id"][-8:])
                for part in ("members", "tabs"):
                    d, f = get_all(f"/teams/{t['id']}/channels/{c['id']}/{part}")
                    if f:
                        not_possible[f"{t['displayName']}/{c['displayName']} {part}"] = {"status": int(f.split()[0]), "message": f[:200]}
                    else:
                        write_json(os.path.join(cd, part + ".json"), d)
                ms, f = get_all(f"/teams/{t['id']}/channels/{c['id']}/messages?$top=50")
                if f:
                    note(f"{t['displayName']}/{c['displayName']} messages", "messages", f)
                for m in ms:
                    rp, f = get_all(f"/teams/{t['id']}/channels/{c['id']}/messages/{m['id']}/replies?$top=50")
                    if f:
                        note(f"replies {m['id']}", "replies", f)
                    m["_replies"] = rp; n["replies"] += len(rp)
                n["messages"] += len(ms); write_json(os.path.join(cd, "messages.json"), ms)
                files, f = get_all(f"/teams/{t['id']}/channels/{c['id']}/filesFolder")
                if not f:
                    write_json(os.path.join(cd, "filesFolder.json"), files)
            open(os.path.join(td, "done"), "w").write("1")
        return n

    def do_chats():
        seen = {}; n = {"chats": 0, "messages": 0, "users_without_chats": []}
        for u in auth.read_users(out):
            d, f = get_all(f"/users/{u['id']}/chats?$top=50")
            if f:
                n["users_without_chats"].append({"upn": u["userPrincipalName"], "message": f[:150]})
                if not f.startswith("404"):
                    note(f"chats {u['userPrincipalName']}", "users/chats", f)
                continue
            for c in d:
                seen.setdefault(c["id"], c)
        write_json(os.path.join(root, "chats/chats.json"), list(seen.values()))
        for cid, c in seen.items():
            cd = os.path.join(root, "chats", cid[:40].replace(":", "_").replace("@", "_"))
            mf = os.path.join(cd, "messages.json")
            if os.path.exists(mf):
                n["chats"] += 1; n["messages"] += len(json.load(open(mf))); continue
            write_json(os.path.join(cd, "chat.json"), c)
            mem, f = get_all(f"/chats/{cid}/members")
            if not f:
                write_json(os.path.join(cd, "members.json"), mem)
            ms, f = get_all(f"/chats/{cid}/messages?$top=50")
            if f:
                note(f"chat {cid[:20]} messages", "chats/messages", f); continue
            write_json(mf, ms); n["chats"] += 1; n["messages"] += len(ms)
        return n

    os.makedirs(root, exist_ok=True)
    a = do_teams(); b = do_chats()
    write_json(os.path.join(root, "_summary.json"), {"teams": a, "chats": b})
    if not os.path.exists(os.path.join(root, "not_possible.json")):
        write_json(os.path.join(root, "not_possible.json"), {})
    auth.log(out, f"done {a} {b['chats']} chats {b['messages']} msgs", "teams")
    return 0


if __name__ == "__main__":
    sys.exit(main())
backup/teams_settings.ps1PowerShell · 2,3 kB
<#
.SYNOPSIS
Sichert die Teams-Einstellungen (Richtlinien, Konfiguration) über das PowerShell-Modul MicrosoftTeams, nur mit Get-Cmdlets.

.DESCRIPTION
Meldet sich mit der App-Registrierung und dem Zertifikat an und führt alle Get-Cs*-Cmdlets des Moduls aus
(ohne Sitzungs-, Voicemail- und einzelne grosse Abfragen). Je Cmdlet entsteht eine JSON-Datei, dazu ein Bericht.
Schlägt die Anmeldung fehl, steht der Grund in _sign_in.json und das Skript endet ohne Fehlercode.

Umgebungsvariablen: M365_CLIENT_ID, M365_TENANT_ID, M365_CERT_PATH (Zertifikat, PEM), M365_CERT_KEY_PATH (Schlüssel, PEM).

.PARAMETER OutDir
Zielordner, normalerweise <backup>/teams-settings.

.EXAMPLE
./teams_settings.ps1 -OutDir ./backup/contoso/20261116-0557/teams-settings
#>
param([Parameter(Mandatory)][string]$OutDir)

$ErrorActionPreference = 'Stop'
foreach ($name in 'M365_CLIENT_ID', 'M365_TENANT_ID', 'M365_CERT_PATH', 'M365_CERT_KEY_PATH') {
    if (-not (Get-Item "env:$name" -ErrorAction SilentlyContinue)) { throw "environment variable $name is not set" }
}
New-Item -ItemType Directory -Force -Path $OutDir | Out-Null
$cert = [System.Security.Cryptography.X509Certificates.X509Certificate2]::CreateFromPemFile($env:M365_CERT_PATH, $env:M365_CERT_KEY_PATH)
$report = @{}
try {
    Connect-MicrosoftTeams -ApplicationId $env:M365_CLIENT_ID -TenantId $env:M365_TENANT_ID -Certificate $cert | Out-Null
    "connected"
}
catch {
    $m = $_.Exception.Message
    "SIGN-IN ERROR: $m"
    @{ sign_in_error = $m } | ConvertTo-Json | Set-Content "$OutDir/_sign_in.json"
    exit 0
}
$cmdlets = (Get-Command -Module MicrosoftTeams -Verb Get |
    Where-Object { $_.Name -like 'Get-Cs*' -and $_.Name -notmatch 'Session|Voicemail|UserPolicyPackageRecommendation|Recording|CallQueueDetails|CsOnlineUser$|SkypeFrequentlyUsed' } |
    ForEach-Object Name | Sort-Object -Unique)
foreach ($c in $cmdlets) {
    try {
        $r = @(& $c -ErrorAction Stop)
        $r | ConvertTo-Json -Depth 8 -WarningAction SilentlyContinue | Set-Content "$OutDir/$($c.Substring(4)).json"
        $report[$c] = @{ count = $r.Count }
        "OK $c $($r.Count)"
    }
    catch {
        $m = $_.Exception.Message
        if ($m.Length -gt 300) { $m = $m.Substring(0, 300) }
        $report[$c] = @{ error = $m }
        "ERROR $c $m"
    }
}
$report | ConvertTo-Json -Depth 4 | Set-Content "$OutDir/_teams_report.json"
backup/verify_files.pyPython · 3,3 kB
"""Vollständigkeitsprüfung: Inventar gegen Manifest, dazu 20 Stichproben mit SHA-256 (nur lokal, kein Netz).
Schreibt graph/completeness_check.json."""
import hashlib, json, os, random, re, sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import m365_auth as auth


def sanitize(s):
    return re.sub(r'[\x00-\x1f<>:"|?*\\]', "_", s).strip() or "_"


def main(argv=None):
    p = auth.make_parser("Check the downloaded files against the inventory and manifest (local only)")
    args = p.parse_args(argv)
    B = args.out
    inventory = json.load(open(B + "/graph/drives_inventory.json"))
    manifest = {}
    for line in open(B + "/manifest.jsonl"):
        j = json.loads(line); manifest[j["path"]] = j
    files, folders = [], 0
    for did, dr in inventory.items():
        items = {i["id"]: i for i in dr["items"]}
        root = next((i["id"] for i in dr["items"] if i.get("name") == "root" and "folder" in i), None)
        m = re.search(r"https://[^/]+/(?:sites|teams|personal)/([^/]+)", dr.get("webUrl") or "")
        site = sanitize(m.group(1) if m else (dr.get("webUrl") or did)[:40]); lib = sanitize(dr.get("name") or did[-8:])

        def rel_path(i, items=items, root=root):
            parts = []
            while i and i in items and i != root:
                parts.append(sanitize(items[i]["name"])); i = items[i].get("parentReference", {}).get("id")
            return "/".join(reversed(parts))
        for it in dr["items"]:
            if "file" in it:
                files.append((f"{site}/{lib}/{rel_path(it['id'])}", it, did))
            elif "folder" in it:
                folders += 1
    res = {"inventory_entries": sum(len(v["items"]) for v in inventory.values()), "inventory_files": len(files),
           "inventory_folders_incl_root": folders, "manifest_lines": sum(1 for _ in open(B + "/manifest.jsonl")),
           "manifest_unique": len(manifest)}
    missing, size_mismatch, paths = [], [], {}
    for rel, it, did in files:
        paths.setdefault(rel, []).append(it["id"])
        target = B + "/files/" + rel
        if rel not in manifest or not os.path.exists(target):
            missing.append(rel)
        elif os.path.getsize(target) != it.get("size"):
            size_mismatch.append((rel, "size", os.path.getsize(target), it.get("size")))
    res["path_collisions"] = {k: v for k, v in paths.items() if len(v) > 1}
    res["missing"] = missing; res["size_mismatch"] = size_mismatch
    random.seed(16)
    sample = random.sample(files, min(20, len(files))); sres = []
    for rel, it, did in sample:
        target = B + "/files/" + rel
        h = hashlib.sha256(open(target, "rb").read()).hexdigest()
        hashes = it["file"].get("hashes") or {}
        sres.append({"path": rel, "size_local": os.path.getsize(target), "size_inventory": it.get("size"), "sha256_local": h,
                     "sha256_manifest": manifest.get(rel, {}).get("sha256"), "hash_in_inventory": hashes.get("sha256Hash") or hashes.get("quickXorHash"),
                     "ok": os.path.getsize(target) == it.get("size") and h == manifest.get(rel, {}).get("sha256")})
    res["sample"] = sres
    auth.write_json(B + "/graph/completeness_check.json", res)
    print({k: (v if not isinstance(v, (list, dict)) else len(v)) for k, v in res.items()}, "sample ok:", sum(s["ok"] for s in sres))
    return 0


if __name__ == "__main__":
    sys.exit(main())
m365_auth.pyPython · 7,8 kB
"""Anmeldung und gemeinsame Helfer für alle Backup-Skripte (nur lesender Zugriff).

Die Anmeldung läuft per App-Registrierung mit Zertifikat (client credentials).
Alle Werte kommen aus Umgebungsvariablen, im Code steht keine feste Kennung:

  M365_TENANT_ID         Verzeichnis-ID des Tenants (<TENANT-ID>)
  M365_CLIENT_ID         Anwendungs-ID der App-Registrierung (<CLIENT-ID>)
  M365_CERT_THUMBPRINT   SHA-1-Fingerabdruck des Zertifikats (<CERT-THUMBPRINT>), hexadezimal
  M365_CERT_KEY_PATH     Pfad zum privaten Schlüssel (PEM); ersatzweise M365_CERT_PATH
  M365_CERT_PATH         Pfad zum Zertifikat (PEM), wird nur von den PowerShell-Skripten gebraucht
  M365_TENANT_NAME       Tenant-Domäne, z. B. contoso.onmicrosoft.com
  M365_ANCHOR_GUID       optional: GUID der System-Mailbox für den Header X-AnchorMailbox
  M365_COMPLIANCE_HOST   Host des Compliance-Endpunkts (nur für die REST-Variante)

Die Module requests und jwt (PyJWT, mit cryptography) werden erst beim ersten Zugriff geladen.
So laufen `--help` und die Unittests auch ohne Anmeldung.
"""
import argparse, base64, json, os, re, threading, time, uuid
from pathlib import Path

GRAPH_SCOPE = "https://graph.microsoft.com/.default"
GRAPH_V1 = "https://graph.microsoft.com/v1.0"
GRAPH_BETA = "https://graph.microsoft.com/beta"
RETRY_CODES = (429, 500, 502, 503, 504)


class ConfigError(Exception):
    """Eine nötige Umgebungsvariable fehlt."""


class InvokeError(Exception):
    """Ein Exchange- oder Compliance-Cmdlet ist fehlgeschlagen."""


def env(name, required=True, default=None):
    value = os.environ.get(name, default)
    if required and not value:
        raise ConfigError(f"environment variable {name} is not set")
    return value


def tenant_prefix():
    """Teil vor .onmicrosoft.com, z. B. 'contoso'."""
    return env("M365_TENANT_NAME").split(".")[0]


_tokens = {}


def token(scope=GRAPH_SCOPE):
    """Zugriffstoken für einen Scope, mit Zertifikat-JWT (RS256) geholt und zwischengespeichert."""
    cached = _tokens.get(scope)
    if cached and time.time() - cached[1] < 2400:
        return cached[0]
    import jwt, requests   # erst hier laden, damit --help ohne Pakete läuft
    tenant, client = env("M365_TENANT_ID"), env("M365_CLIENT_ID")
    x5t = base64.urlsafe_b64encode(bytes.fromhex(env("M365_CERT_THUMBPRINT"))).decode().rstrip("=")
    key_path = env("M365_CERT_KEY_PATH", required=False) or env("M365_CERT_PATH")
    now = int(time.time())
    url = f"https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token"
    assertion = jwt.encode({"aud": url, "iss": client, "sub": client, "jti": str(uuid.uuid4()),
                            "nbf": now, "exp": now + 600},
                           Path(key_path).read_text(), algorithm="RS256", headers={"x5t": x5t})
    r = requests.post(url, data={
        "client_id": client, "grant_type": "client_credentials", "scope": scope,
        "client_assertion_type": "urn:ietf:params:oauth:client-assertion-type:jwt-bearer",
        "client_assertion": assertion}, timeout=30)
    j = r.json()
    if "access_token" not in j:
        raise RuntimeError(f"sign-in failed: {j.get('error')}: {str(j.get('error_description', ''))[:150]}")
    _tokens[scope] = (j["access_token"], time.time())
    return j["access_token"]


def g(method, path, scope=GRAPH_SCOPE, **kw):
    """Graph-Aufruf mit Wiederholung bei 429/5xx und Verbindungsfehlern. Gibt die Response zurück."""
    import requests
    url = path if path.startswith("http") else GRAPH_V1 + path
    extra = kw.pop("headers", {})
    r = None
    for attempt in range(8):
        try:
            r = requests.request(method, url, headers={"Authorization": "Bearer " + token(scope), **extra},
                                 timeout=60, **kw)
        except requests.RequestException:
            time.sleep(5 * (attempt + 1))
            continue
        if r.status_code not in RETRY_CODES:
            return r
        time.sleep(min(int(r.headers.get("Retry-After", 5 * (attempt + 1))), 120) + 1)
    if r is None:
        raise RuntimeError("Graph is not reachable")
    return r


def get_all(path, beta=False):
    """GET mit Paging. Gibt (liste, None) oder (teilliste, 'status text') zurück.
    Liefert der Endpunkt ein Einzelobjekt (ohne 'value'), kommt es als Liste mit einem Element."""
    out = []
    url = path if path.startswith("http") else (GRAPH_BETA if beta else GRAPH_V1) + path
    while url:
        r = g("GET", url)
        if r.status_code != 200:
            return out, f"{r.status_code} {r.text[:300]}"
        try:
            j = r.json()
        except Exception:
            return out, f"{r.status_code} no JSON answer (empty)"
        if "value" not in j:
            return [j], None
        out += j["value"]
        url = j.get("@odata.nextLink")
    return out, None


def invoke(cmd, params=None, host="outlook.office365.com", scope=None, anchor=None):
    """Ein Get-Cmdlet über die Exchange-Admin-REST-Schnittstelle, mit Paging. Gibt eine Liste zurück.
    Nur Get-Cmdlets sind erlaubt (Sicherung gegen versehentliche Änderungen)."""
    import requests
    assert cmd.startswith("Get-"), cmd
    tenant = env("M365_TENANT_ID")
    scope = scope or f"https://{host}/.default"
    url = f"https://{host}/adminapi/beta/{tenant}/InvokeCommand"
    guid = env("M365_ANCHOR_GUID", required=False)
    anchor = anchor or (f"APP:SystemMailbox{{{guid}}}@{tenant}" if guid else None)
    out = []
    body = {"CmdletInput": {"CmdletName": cmd, "Parameters": params or {}}}
    while url:
        for attempt in range(4):
            headers = {"Authorization": "Bearer " + token(scope), "Content-Type": "application/json",
                       "Prefer": "odata.maxpagesize=1000", "Accept-Encoding": "identity"}
            if anchor:
                headers["X-AnchorMailbox"] = anchor
            r = requests.post(url, json=body, timeout=300, headers=headers)
            if r.status_code in (429, 502, 503, 504):
                time.sleep(5 * (attempt + 1))
                continue
            break
        if r.status_code != 200:
            try:
                message = r.json()["error"]["message"]
            except Exception:
                message = r.text
            raise InvokeError(f"HTTP {r.status_code}: {message[:400]}")
        j = r.json()
        out += j.get("value", [])
        url = j.get("@odata.nextLink")
    return out


# ---------------------------------------------------------------- gemeinsame Helfer

_lock = threading.Lock()


def log(out_dir, text, area=""):
    """Zeile in <out>/backup.log anhängen (threadsicher)."""
    with _lock:
        with open(os.path.join(out_dir, "backup.log"), "a") as f:
            f.write(time.strftime("%H:%M:%S ") + (f"[{area}] " if area else "") + text + "\n")


def safe_name(s):
    """Dateisystem-sicherer Name (höchstens 80 Zeichen)."""
    return re.sub(r"[^A-Za-z0-9._@-]+", "_", s or "x")[:80]


def write_json(path, obj):
    """JSON atomar schreiben (erst .tmp, dann ersetzen)."""
    os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
    tmp = path + ".tmp"
    with open(tmp, "w") as f:
        json.dump(obj, f, ensure_ascii=False, indent=1, default=str)
    os.replace(tmp, path)


def read_users(out_dir):
    """Benutzerliste aus dem Graph-Export (graph/users.json im Zielordner)."""
    with open(os.path.join(out_dir, "graph", "users.json")) as f:
        return json.load(f)


def folder_size(path):
    """(Anzahl Dateien, Summe Bytes) unter einem Ordner."""
    n = total = 0
    for d, _, files in os.walk(path):
        for name in files:
            n += 1
            total += os.path.getsize(os.path.join(d, name))
    return n, total


def make_parser(description):
    """Gemeinsamer Argumentparser: --out ist der Zielordner des Backups."""
    p = argparse.ArgumentParser(description=description)
    p.add_argument("--out", required=True, help="backup target folder (read-only access to the tenant)")
    return p
restore/secrets_restore.pyPython · 31,8 kB
#!/usr/bin/env python3
"""Restore-Hilfe für Secrets: Kennwörter, App-Secrets und Zertifikate.

Ein Backup enthält keine Secret-Werte. Dieses Skript liest den Plan aus dem Backup, legt im ZIEL-Tenant
neue Secrets an und sichert die neuen Werte in einem Speicher (Datei mit Rechten 600, optional Azure Key Vault).
Die nicht gesicherten Kennwörter und Secret-Werte werden interaktiv abgefragt (getpass, Eingabe unsichtbar).
Ohne --execute gibt es nur einen Trockenlauf (kein Netz, keine Änderung).
Nur Standardbibliothek plus requests. Werte erscheinen nie in Log, Journal oder Ausgabe.
"""
import argparse, base64, getpass, hashlib, json, os, re, secrets, string, subprocess
import sys, tempfile, time, unittest, uuid
from datetime import datetime, timezone, timedelta
from pathlib import Path

PROGRESS_DEFAULT = "restore_secrets_progress.json"
STORE_DEFAULT = "restore_secrets_store.json"
GUID = re.compile(r"^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$")


# ----------------------------------------------------------------- Hilfen


def now(): return datetime.now(timezone.utc)
def iso(d): return d.strftime("%Y-%m-%dT%H:%M:%SZ")


def load_json(path, default):
    p = Path(path)
    if not p.exists():
        return default
    return json.loads(p.read_text(encoding="utf-8"))


def write_600(path, data):
    """Atomar mit Rechten 600 schreiben (die Datei ist nie für andere lesbar)."""
    p = Path(path); tmp = p.with_name(p.name + ".tmp")
    fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
    with os.fdopen(fd, "w", encoding="utf-8") as f:
        json.dump(data, f, indent=1, ensure_ascii=False)
    os.chmod(tmp, 0o600); os.replace(tmp, p)

# ---------------------------------------------------------- Kennwortregeln


def check_password(pw, upn=""):
    """Gibt die Liste der Verstösse zurück (leer = in Ordnung). Strenger als Entra (8-256, 3 von 4 Klassen)."""
    f = []
    if len(pw) < 12: f.append("at least 12 characters")
    if len(pw) > 256: f.append("at most 256 characters")
    classes = sum([any(c.islower() for c in pw), any(c.isupper() for c in pw),
                   any(c.isdigit() for c in pw), any(not c.isalnum() and c.isascii() for c in pw)])
    if classes < 3: f.append("at least 3 of 4: upper case, lower case, digits, special characters")
    if pw != pw.strip(): f.append("no space at the start or end")
    if not pw.isascii(): f.append("ASCII characters only")
    local = upn.split("@")[0].lower()
    if len(local) >= 3 and local in pw.lower(): f.append("must not contain the user name")
    return f


def random_password(length=20):
    chars = string.ascii_letters + string.digits + "!#$%&*+-=?@_"
    while True:
        pw = "".join(secrets.choice(chars) for _ in range(length))
        if not check_password(pw) and all(any(c in s for c in pw) for s in
                                          (string.ascii_lowercase, string.ascii_uppercase, string.digits, "!#$%&*+-=?@_")):
            return pw


def map_upn(upn, new_domain=None):
    """UPN im Zieltenant: Domäne ersetzen (Option --new-domain), sonst unverändert."""
    if not new_domain or "@" not in upn:
        return upn
    return upn.rsplit("@", 1)[0] + "@" + new_domain.lstrip("@")

# ------------------------------------------------------------------- Plan


def read_inventory(backup):
    """Inventar der Secrets lesen (secrets_inventory.json im Backup-Ordner)."""
    f = Path(backup) / "secrets_inventory.json"
    return json.loads(f.read_text(encoding="utf-8")) if f.exists() else []


def read_plan(backup):
    b = Path(backup); g = b / "graph"

    def j(name, default):
        p = g / name
        return json.loads(p.read_text(encoding="utf-8")) if p.exists() else default
    users = j("users.json", []); apps = j("applications.json", []); sps = j("servicePrincipals.json", [])
    inventory = read_inventory(backup)
    plan = {"users": [], "app_secrets": [], "app_certificates": [], "sp_credentials": [], "notes": []}
    guests = synced = 0
    for u in users:
        upn = u.get("userPrincipalName", "")
        if u.get("userType") == "Guest" or "#EXT#" in upn:
            guests += 1; continue
        if u.get("onPremisesSyncEnabled"):
            synced += 1; continue
        plan["users"].append({"id": u["id"], "upn": upn, "name": u.get("displayName", "")})

    def collect(objs, kind, target_pw, target_key):
        for o in objs:
            for k in o.get("passwordCredentials") or []:
                target_pw.append({"objectId": o["id"], "appId": o.get("appId"), "app": o.get("displayName"),
                                  "keyId": k.get("keyId"), "display_name": k.get("displayName") or "",
                                  "expires": k.get("endDateTime"), "type": kind})
            for k in o.get("keyCredentials") or []:
                target_key.append({"objectId": o["id"], "appId": o.get("appId"), "app": o.get("displayName"),
                                   "keyId": k.get("keyId"), "display_name": k.get("displayName") or "",
                                   "expires": k.get("endDateTime"), "usage": k.get("usage"),
                                   "key_type": k.get("type"), "type": kind})
    collect(apps, "application", plan["app_secrets"], plan["app_certificates"])
    collect(sps, "servicePrincipal", plan["sp_credentials"], plan["sp_credentials"])
    for e in plan["sp_credentials"]:
        e.setdefault("kind", "Certificate" if "usage" in e else "Secret")
    # Inventar nur zum Abgleich: Einträge, die in graph/ fehlen, ergänzen
    known = {(e["objectId"], e["keyId"]) for k in ("app_secrets", "app_certificates", "sp_credentials") for e in plan[k]}
    missing = 0
    for e in inventory:
        if (e.get("objectId"), e.get("keyId")) in known:
            continue
        missing += 1
        kind = e.get("kind") or ""
        entry = {"objectId": e.get("objectId"), "appId": e.get("appId"), "app": e.get("app"), "keyId": e.get("keyId"),
                 "display_name": e.get("display_name") or "", "expires": e.get("expires"), "type": "application",
                 "usage": e.get("usage"), "key_type": e.get("key_type")}
        if e.get("source") == "servicePrincipals":
            entry["type"] = "servicePrincipal"; plan["sp_credentials"].append(dict(entry, kind=kind))
        elif kind.startswith("Secret"):
            plan["app_secrets"].append(entry)
        else:
            plan["app_certificates"].append(entry)
    plan["_inventory_added"] = missing
    # Hinweisliste (Checkliste)
    grants = len(j("oauth2PermissionGrants.json", []))
    notes = plan["notes"]

    def note(i, t): notes.append({"id": i, "text": t})
    note("mfa", "MFA methods (Authenticator, FIDO2, phone) and Windows Hello: every person registers again; issue a Temporary Access Pass if needed.")
    note("break-glass", "Emergency accounts (break glass): set a long password of their own, store it offline in a safe, test the sign-in.")
    if guests: note("guests", f"{guests} guest account(s) are not set: invite them again in the target tenant.")
    if synced: note("sync", f"{synced} synchronized account(s): the password comes from the local AD (Entra Connect), do not set it.")
    if grants: note("consent", f"{grants} permission grant(s) (oauth2PermissionGrants): grant admin consent per app again in the target tenant; afterwards replace the app secrets in the systems that use them.")
    note("consumers", "Update every system that needs a new app secret or certificate (scripts, pipelines, services) with the new values.")
    note("connections", "Re-authenticate connections/connectors (Power Automate, Logic Apps, Power Apps, Teams connectors, webhooks); connections belong to people and are not backed up.")
    note("service-accounts", "Enter service accounts and passwords in scripts, task scheduler, automation runbooks, pipelines and configurations (for example M365DSC, Exchange automation).")
    note("exchange", "Exchange Online: create a new certificate for the automation app (app-only) and store it in the script; check SMTP relay, connectors and app passwords.")
    note("sharepoint", "SharePoint/Teams: set up sharing links, webhooks and service accounts for synchronization (for example Intune, third parties) again.")
    note("sspr", "Self-service password reset: let people register their contact data and methods again.")
    note("mail-oauth", "Third-party applications with SAML/SSO (Salesforce, Box, LinkedIn and so on): link certificates and secrets in the third-party system again.")
    note("licenses", "Assign licenses before the first sign-in, otherwise mailbox and services are missing.")
    return plan


def print_plan(plan, out=print):
    out("== Plan: places that need a secret ==")
    out(f"(a) user passwords                : {len(plan['users'])}")
    out(f"(b) app secrets                   : {len(plan['app_secrets'])}")
    out(f"(c) app certificates              : {len(plan['app_certificates'])}")
    out(f"(d) service principal credentials : {len(plan['sp_credentials'])}")
    out(f"(e) notes (checklist, manual)     : {len(plan['notes'])}")
    if plan.get("_inventory_added"):
        out(f"    (added from the secrets inventory: {plan['_inventory_added']})")
    out("\n(a) Users:")
    for u in plan["users"]:
        out(f"   - {u['upn']}  ({u['name']})")
    for t, n in (("b", "app_secrets"), ("c", "app_certificates"), ("d", "sp_credentials")):
        out(f"\n({t}) {n}:")
        for e in plan[n]:
            out(f"   - {e['app']} [{e['type']}] {e.get('kind') or ''} '{e['display_name']}' keyId={e['keyId']} expires={e['expires']}")
    out("\n(e) Checklist:")
    for h in plan["notes"]:
        out(f"   [ ] {h['text']}")

# ------------------------------------------------------------- Journal


PROGRESS_FIELDS = ("id", "category", "object", "kind", "expires", "keyId", "time", "status")


class Progress:
    """Hält nur Felder der Whitelist; Werte können gar nicht hineingelangen."""
    def __init__(self, path):
        self.path = Path(path)
        self.d = load_json(path, {"version": 1, "entries": [], "checklist": {}})

    def done(self, eid): return any(e["id"] == eid and e["status"] == "ok" for e in self.d["entries"])

    def add(self, **kw):
        e = {k: kw.get(k) for k in PROGRESS_FIELDS}; e["time"] = iso(now())
        self.d["entries"] = [x for x in self.d["entries"] if x["id"] != e["id"]] + [e]
        self.save()

    def save(self): write_600(self.path, self.d)

# ----------------------------------------------------------------- Speicher


class Store:
    """Neue Werte: Datei (600), Bildschirm (einmalig am Ende) und/oder Key Vault."""
    def __init__(self, mode, path, kv=None):
        self.mode, self.path, self.kv = mode, Path(path), kv
        self.screen = {}
        self.file = load_json(path, {}) if mode == "file" else {}

    def put(self, name, value, meta):
        if self.mode == "file":
            self.file[name] = dict(meta, value=value); write_600(self.path, self.file)
        else:
            self.screen[name] = dict(meta, value=value)
        if self.kv: self.kv(name, value, meta)

    def finish(self, interactive):
        if self.mode == "file" and self.file:
            print(f"\nNew values are stored in {self.path} (permissions 600).")
            print("IMPORTANT: move them into a password manager or Azure Key Vault, then DELETE the file (shred -u).")
        if self.mode == "screen" and self.screen:
            if interactive and input("\nShow the new values ONCE on screen now? (y/n) ").strip().lower() == "y":
                for k, v in self.screen.items(): print(f"  {k}: {v['value']}")
                print("Save them now; they are not shown again.")
            else:
                print("Values were not shown and are lost (mode screen).")

# ------------------------------------------------------------------ Graph


class RestoreError(Exception): pass


class Target:
    """Zugang zum ZIEL-Tenant. Wird nur im Ausführungsmodus benutzt."""
    def __init__(self, tenant, client_id, live):
        self.tenant, self.client_id, self.live = tenant, client_id, live
        self._t = {}

    def _assertion(self, pem, key):
        der = subprocess.run(["openssl", "x509", "-in", pem, "-outform", "DER"], capture_output=True, check=True).stdout
        x5t = base64.urlsafe_b64encode(hashlib.sha1(der).digest()).rstrip(b"=").decode()
        b64 = lambda b: base64.urlsafe_b64encode(b).rstrip(b"=")
        aud = f"https://login.microsoftonline.com/{self.tenant}/oauth2/v2.0/token"
        head = b64(json.dumps({"alg": "RS256", "typ": "JWT", "x5t": x5t}).encode())
        n = int(time.time())
        body = b64(json.dumps({"aud": aud, "iss": self.client_id, "sub": self.client_id, "jti": str(uuid.uuid4()), "nbf": n, "exp": n + 600}).encode())
        raw = head + b"." + body
        sig = subprocess.run(["openssl", "dgst", "-sha256", "-sign", key], input=raw, capture_output=True, check=True).stdout
        return (raw + b"." + b64(sig)).decode()

    def token(self, scope):
        import requests
        if not self.live: raise RestoreError("network access is blocked in dry-run mode")
        t = self._t.get(scope)
        if t and time.time() < t[1]: return t[0]
        data = {"client_id": self.client_id, "grant_type": "client_credentials", "scope": scope}
        pem, key = os.environ.get("M365_TARGET_CERT_PEM"), os.environ.get("M365_TARGET_CERT_KEY")
        if pem and key:
            data.update(client_assertion_type="urn:ietf:params:oauth:client-assertion-type:jwt-bearer",
                        client_assertion=self._assertion(pem, key))
        else:
            sec = os.environ.get("M365_TARGET_CLIENT_SECRET") or getpass.getpass("Client secret of the target tenant (input is hidden): ")
            os.environ["M365_TARGET_CLIENT_SECRET"] = sec   # nur im eigenen Prozess; nie ausgegeben
            data["client_secret"] = sec
        r = requests.post(f"https://login.microsoftonline.com/{self.tenant}/oauth2/v2.0/token", data=data, timeout=30)
        j = r.json()
        if "access_token" not in j:
            raise RestoreError(f"sign-in failed: {j.get('error')} {str(j.get('error_description', ''))[:120]}")
        self._t[scope] = (j["access_token"], time.time() + int(j.get("expires_in", 3000)) - 120)
        return j["access_token"]

    def g(self, method, path, scope="https://graph.microsoft.com/.default", **kw):
        import requests
        url = path if path.startswith("http") else "https://graph.microsoft.com/v1.0" + path
        r = None
        for v in range(6):
            try:
                r = requests.request(method, url, headers={"Authorization": "Bearer " + self.token(scope)}, timeout=60, **kw)
            except requests.RequestException:
                time.sleep(3 * (v + 1)); continue
            if r.status_code not in (429, 500, 502, 503, 504): return r
            time.sleep(min(int(r.headers.get("Retry-After", 3 * (v + 1))), 60))
        if r is None: raise RestoreError("Graph is not reachable")
        return r

    def find(self, kind, name=None, object_id=None):
        """Objekt im Ziel finden; gibt ein dict mit id oder None zurück."""
        if object_id:
            r = self.g("GET", f"/{kind}/{object_id}")
            return r.json() if r.status_code == 200 else None
        q = name.replace("'", "''")
        r = self.g("GET", f"/{kind}?$filter=displayName eq '{q}'&$select=id,displayName,appId,keyCredentials")
        if r.status_code != 200: raise RestoreError(f"search {kind} '{name}': HTTP {r.status_code}")
        v = r.json().get("value", [])
        if len(v) > 1: raise RestoreError(f"'{name}' is not unique in the target ({len(v)} hits); use --app-mapping")
        return v[0] if v else None


def keyvault_store(target, vault_name):
    def f(name, value, meta):
        sn = re.sub(r"[^0-9A-Za-z-]", "-", name)[:127]
        r = target.g("PUT", f"https://{vault_name}.vault.azure.net/secrets/{sn}?api-version=7.4",
                     scope="https://vault.azure.net/.default", json={"value": value, "attributes": {"enabled": True}, "tags": {"source": "secrets_restore"}})
        if r.status_code not in (200, 201): raise RestoreError(f"Key Vault '{sn}': HTTP {r.status_code}")
    return f

# ------------------------------------------------------------- Zertifikate


def new_cert(app, days, folder):
    """Selbstsigniertes Zertifikat per openssl; privater Schlüssel mit Rechten 600. Gibt (b64_der, key_path, end) zurück."""
    folder = Path(folder); folder.mkdir(parents=True, exist_ok=True)
    os.chmod(folder, 0o700)
    stem = re.sub(r"[^0-9A-Za-z_-]", "_", app) + "_" + uuid.uuid4().hex[:8]
    key, crt = folder / (stem + ".key"), folder / (stem + ".crt")
    old = os.umask(0o077)
    try:
        subprocess.run(["openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", "-keyout", str(key), "-out", str(crt),
                        "-days", str(days), "-subj", "/CN=" + re.sub(r"[^0-9A-Za-z _.-]", "", app)], capture_output=True, check=True)
    finally:
        os.umask(old)
    os.chmod(key, 0o600)
    return cert_b64(crt), str(key), now() + timedelta(days=days)


def cert_b64(path):
    raw = Path(path).read_bytes()
    form = "PEM" if b"-----BEGIN" in raw else "DER"
    der = subprocess.run(["openssl", "x509", "-inform", form, "-in", str(path), "-outform", "DER"], capture_output=True, check=True).stdout
    return base64.b64encode(der).decode()

# ------------------------------------------------------------------ Eingabe


def ask(text, allowed, default=None):
    while True:
        a = input(text).strip().lower() or (default or "")
        if a in allowed: return a
        print("  Please enter one of:", "/".join(allowed))


def enter_password(upn):
    """Kennwort zweimal abfragen (getpass, unsichtbar); es wird nie ausgegeben oder protokolliert."""
    while True:
        a = getpass.getpass(f"  Password for {upn}: "); b = getpass.getpass("  Again: ")
        if a != b:
            print("  The entries differ."); continue
        f = check_password(a, upn)
        if f:
            print("  Too weak: " + "; ".join(f)); continue
        return a

# --------------------------------------------------------------- Ausführung


def eid(*parts): return ":".join(str(t) for t in parts)


def target_object(target, kind, e, mapping):
    z = mapping.get(e["appId"]) or mapping.get(e["objectId"]) or mapping.get(e["app"])
    if z and GUID.match(z): o = target.find(kind, object_id=z)
    else: o = target.find(kind, name=z or e["app"])
    if not o: raise RestoreError(f"'{e['app']}' not found in the target (use a mapping file or create the app first)")
    return o


def run(args, plan, answers, interactive):
    live = args.execute
    target = Target(args.tenant, args.client_id, live)
    progress = Progress(args.progress_file)
    kv = keyvault_store(target, args.keyvault) if args.keyvault else None
    store = Store(args.output, args.store_file, kv)
    mapping = json.loads(Path(args.app_mapping).read_text(encoding="utf-8")) if args.app_mapping else {}
    errors = 0

    def answer(area, key, default=None):
        b = answers.get(area, {})
        return b.get(key, b.get("default", default))
    # ---- Schritt 2: Benutzer
    print("\n== Step 2: users ==")
    all_random = args.all_random or answers.get("users", {}).get("default") == "random"
    for u in plan["users"]:
        upn = map_upn(u["upn"], args.new_domain); pid = eid("user", upn.lower())
        if progress.done(pid):
            print(f"  skipped (done): {upn}"); continue
        mode = answer("users", u["upn"])
        if not mode:
            if all_random: mode = "random"
            elif interactive:
                a = ask(f"  {upn}: [r]andom / [e]nter / [a]ll remaining random / [s]kip: ", ("r", "e", "a", "s"), "r")
                if a == "a": all_random = True; a = "r"
                mode = {"r": "random", "e": "enter", "s": "skip"}[a]
            elif not live: mode = "random"   # Trockenlauf: nur den Standard anzeigen
            else: raise RestoreError(f"no decision for {u['upn']} (--answers, --all-random or interactive)")
        if mode == "skip":
            print(f"  skipped: {upn}"); continue
        if mode == "enter" and not interactive and not sys.stdin.isatty(): raise RestoreError("password entry needs a terminal")
        if not live:
            print(f"  [dry run] {upn}: mode {mode}, change at first sign-in"); continue
        try:
            t = target.find("users", object_id=upn)
            if not t: raise RestoreError(f"user {upn} not found in the target")
            pw = random_password() if mode == "random" else enter_password(upn)
            r = target.g("PATCH", f"/users/{t['id']}", json={"passwordProfile": {"password": pw, "forceChangePasswordNextSignIn": True}})
            if r.status_code not in (200, 204): raise RestoreError(f"PATCH {upn}: HTTP {r.status_code}")
            store.put(f"user-{upn}", pw, {"kind": "user password", "object": upn})
            progress.add(id=pid, category="user", object=upn, kind="password(" + mode + ")", status="ok")
            print(f"  set: {upn}")
        except RestoreError as e:
            errors += 1; print(f"  ERROR {upn}: {e}")
            progress.add(id=pid, category="user", object=upn, kind="password", status="error")
    # ---- Schritt 3: Secrets (Apps und Dienstprinzipale)
    print("\n== Step 3: app secrets / service principal secrets ==")
    secret_list = [("applications", e) for e in plan["app_secrets"]] + \
                  [("servicePrincipals", e) for e in plan["sp_credentials"] if (e.get("kind") or "Secret").startswith("Secret")]
    for kind, e in secret_list:
        pid = eid("secret", kind, e["objectId"], e["keyId"])
        if progress.done(pid):
            print(f"  skipped (done): {e['app']} '{e['display_name']}'"); continue
        days = int(answer("app_secrets", e["keyId"]) or answers.get("lifetime_days") or args.lifetime)
        if interactive and live and not answers.get("app_secrets") and not args.lifetime_fixed:
            a = input(f"  {e['app']} '{e['display_name']}': lifetime in days [{days}]: ").strip()
            if a.isdigit(): days = int(a)
        if not 1 <= days <= 730: raise RestoreError("lifetime must be 1 to 730 days")
        if not live:
            print(f"  [dry run] {kind[:-1]} {e['app']}: new secret '{e['display_name']}', {days} days"); continue
        try:
            o = target_object(target, kind, e, mapping)
            end = now() + timedelta(days=days)
            r = target.g("POST", f"/{kind}/{o['id']}/addPassword", json={"passwordCredential": {
                "displayName": e["display_name"] or "restore", "endDateTime": iso(end)}})
            if r.status_code not in (200, 201): raise RestoreError(f"addPassword {e['app']}: HTTP {r.status_code}")
            j = r.json()
            store.put(f"secret-{e['app']}-{j['keyId'][:8]}", j["secretText"],
                      {"kind": "secret", "object": e["app"], "appId": o.get("appId"), "expires": j.get("endDateTime")})
            progress.add(id=pid, category="secret", object=e["app"], kind="secret", expires=j.get("endDateTime"), keyId=j["keyId"], status="ok")
            print(f"  created: {e['app']} '{e['display_name']}' until {j.get('endDateTime')}")
        except RestoreError as x:
            errors += 1; print(f"  ERROR {e['app']}: {x}")
            progress.add(id=pid, category="secret", object=e["app"], kind="secret", status="error")
    # ---- Schritt 4: Zertifikate
    print("\n== Step 4: certificates ==")
    certs = [("applications", e) for e in plan["app_certificates"]] + \
            [("servicePrincipals", e) for e in plan["sp_credentials"] if not (e.get("kind") or "Secret").startswith("Secret")]
    for kind, e in certs:
        pid = eid("cert", kind, e["objectId"], e["keyId"])
        if progress.done(pid):
            print(f"  skipped (done): {e['app']} '{e['display_name']}'"); continue
        a = answer("certificates", e["keyId"])
        if not a:
            if interactive: a = ({"n": "new", "f": "file", "s": "skip"})[ask(f"  {e['app']} '{e['display_name']}': [n]ew self-signed / upload [f]ile (.cer/.pem) / [s]kip: ", ("n", "f", "s"), "n")]
            else: a = "new"
        path = None
        if isinstance(a, dict): path = a.get("file"); a = "file"
        if a == "file" and not path:
            path = input("  Path of the .cer/.pem file: ").strip() if interactive else None
            if not path: raise RestoreError("certificate file is missing")
        if a == "skip":
            print(f"  skipped: {e['app']}"); continue
        if not live:
            print(f"  [dry run] {e['app']}: attach {'file ' + path if path else 'a new self-signed certificate'}"); continue
        try:
            o = target_object(target, kind, e, mapping)
            if path: b64, key_path, end = cert_b64(path), None, now() + timedelta(days=365)
            else: b64, key_path, end = new_cert(e["app"], int(answers.get("certificate_days", 365)), args.key_folder)
            r0 = target.g("GET", f"/{kind}/{o['id']}?$select=keyCredentials")
            existing = [{k: v for k, v in kc.items() if k in ("customKeyIdentifier", "displayName", "endDateTime", "key", "keyId", "startDateTime", "type", "usage")}
                        for kc in r0.json().get("keyCredentials", [])] if r0.status_code == 200 else []
            new = {"type": "AsymmetricX509Cert", "usage": e.get("usage") or "Verify", "key": b64,
                   "displayName": e["display_name"] or "restore", "endDateTime": iso(end), "startDateTime": iso(now())}
            if not path:   # Zeitfenster aus dem Zertifikat lesen ist optional; Graph prüft selbst
                new.pop("endDateTime"); new.pop("startDateTime")
            r = target.g("PATCH", f"/{kind}/{o['id']}", json={"keyCredentials": existing + [new]})
            if r.status_code not in (200, 204): raise RestoreError(f"PATCH keyCredentials {e['app']}: HTTP {r.status_code}")
            if key_path: store.put(f"certificate-{e['app']}-{Path(key_path).stem[-8:]}", f"Private key: file {key_path}", {"kind": "certificate", "object": e["app"]})
            progress.add(id=pid, category="certificate", object=e["app"], kind="certificate", expires=iso(end), status="ok")
            print(f"  attached: {e['app']} '{e['display_name']}'" + (f" (key: {key_path})" if key_path else ""))
        except (RestoreError, subprocess.CalledProcessError) as x:
            errors += 1; print(f"  ERROR {e['app']}: {x}")
            progress.add(id=pid, category="certificate", object=e["app"], kind="certificate", status="error")
    # ---- Checkliste
    print("\n== Checklist (manual) ==")
    done = set(answers.get("checklist_done", [])) | {k for k, v in progress.d.get("checklist", {}).items() if v}
    for h in plan["notes"]:
        ok = h["id"] in done
        if not ok and interactive and live:
            ok = input(f"  {h['text']}\n    Done? (y/n) ").strip().lower() == "y"
        print(f"  [{'x' if ok else ' '}] {h['text']}")
        if live: progress.d.setdefault("checklist", {})[h["id"]] = ok
    if live:
        progress.save(); store.finish(interactive)
    else:
        print("\nDRY RUN: nothing sent, nothing written. Use --execute to really restore.")
    return errors

# ---------------------------------------------------------------- Selbsttest


class Tests(unittest.TestCase):
    def test_password(self):
        self.assertTrue(check_password("short1A!"))
        self.assertTrue(check_password("onlylowercaseletters"))
        self.assertTrue(check_password("Jane.Doe123x", "[email protected]"))
        self.assertFalse(check_password("Good-Password-2026"))
        for _ in range(50): self.assertFalse(check_password(random_password()))

    def test_upn(self):
        self.assertEqual(map_upn("[email protected]", "new.example"), "[email protected]")
        self.assertEqual(map_upn("[email protected]"), "[email protected]")
        self.assertEqual(map_upn("[email protected]", "@c.example"), "[email protected]")

    def test_progress_without_values(self):
        with tempfile.TemporaryDirectory() as d:
            p = Progress(Path(d) / "p.json")
            value = "S3cr3t-Value-ZZZ-" + secrets.token_hex(8)
            p.add(id="x", category="secret", object="app", kind="secret", expires="2027", keyId="k", status="ok",
                  value=value, secretText=value)
            text = (Path(d) / "p.json").read_text()
            self.assertNotIn(value, text); self.assertNotIn("value", text)
            self.assertEqual(oct((Path(d) / "p.json").stat().st_mode & 0o777), "0o600")
            self.assertTrue(Progress(Path(d) / "p.json").done("x"))

    def test_store_permissions(self):
        with tempfile.TemporaryDirectory() as d:
            t = Store("file", Path(d) / "t.json"); t.put("a", "w", {"kind": "x"})
            self.assertEqual(oct((Path(d) / "t.json").stat().st_mode & 0o777), "0o600")

    def test_certificate(self):
        with tempfile.TemporaryDirectory() as d:
            b64, key, end = new_cert("Test App", 30, d)
            self.assertTrue(base64.b64decode(b64)); self.assertEqual(oct(Path(key).stat().st_mode & 0o777), "0o600")

    def test_dry_run_blocks_network(self):
        with self.assertRaises(RestoreError): Target("t", "c", False).token("x")

    def test_inventory(self):
        with tempfile.TemporaryDirectory() as d:
            (Path(d) / "secrets_inventory.json").write_text(json.dumps([{"source": "applications", "kind": "Secret"}]))
            self.assertEqual(read_inventory(d)[0]["kind"], "Secret")
            self.assertEqual(read_inventory(d + "/missing"), [])


def self_test():
    r = unittest.TextTestRunner(verbosity=2).run(unittest.defaultTestLoader.loadTestsFromTestCase(Tests))
    return 0 if r.wasSuccessful() else 1

# -------------------------------------------------------------------- main


def main(argv=None):
    ap = argparse.ArgumentParser(description="Restore helper for secrets: dry run by default, --execute really restores")
    ap.add_argument("--backup"); ap.add_argument("--tenant"); ap.add_argument("--client-id")
    ap.add_argument("--execute", action="store_true", help="really restore (otherwise dry run)")
    ap.add_argument("--new-domain", help="new domain for UPNs in the target tenant")
    ap.add_argument("--app-mapping", help="JSON: backup app (name/appId/objectId) -> target (name or objectId)")
    ap.add_argument("--answers", help="JSON with predefined decisions (non-interactive)")
    ap.add_argument("--all-random", action="store_true", help="random passwords for all users")
    ap.add_argument("--lifetime", type=int, default=180, help="lifetime of new secrets in days (default 180)")
    ap.add_argument("--lifetime-fixed", action="store_true", help="do not ask the lifetime per secret")
    ap.add_argument("--output", choices=("file", "screen"), default="file")
    ap.add_argument("--store-file", default=STORE_DEFAULT, help="output file for new values (permissions 600)")
    ap.add_argument("--keyvault", help="name of an Azure Key Vault; also store the values there")
    ap.add_argument("--key-folder", default="./restore_keys", help="folder for new private keys")
    ap.add_argument("--progress-file", default=PROGRESS_DEFAULT)
    ap.add_argument("--self-test", action="store_true")
    a = ap.parse_args(argv)
    if a.self_test: return self_test()
    for n in ("backup", "tenant", "client_id"):
        if not getattr(a, n): ap.error(f"--{n.replace('_', '-')} is missing")
    if not Path(a.backup).is_dir(): ap.error("backup folder not found")
    plan = read_plan(a.backup)
    print(f"Backup : {a.backup}\nTarget : {a.tenant}  (application {a.client_id})")
    print("Mode   : " + ("EXECUTE" if a.execute else "DRY RUN (default, no network)") + "\n")
    print_plan(plan)
    answers = json.loads(Path(a.answers).read_text(encoding="utf-8")) if a.answers else {}
    interactive = not a.answers and sys.stdin.isatty() and a.execute
    if a.execute:
        print("\nNOTE: never copy new values into logs or chats. Sign-in only via environment (M365_TARGET_CLIENT_SECRET or M365_TARGET_CERT_PEM + M365_TARGET_CERT_KEY) or prompt.")
        if interactive and input("Really restore into the target tenant? (yes/no) ").strip().lower() != "yes": return 1
    try:
        errors = run(a, plan, answers, interactive)
    except RestoreError as e:
        print("ABORT:", e); return 2
    return 1 if errors else 0


if __name__ == "__main__":
    sys.exit(main())